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FC3 129 - Interaction Designs For Ubicomp Cultures 

Ulrik Ekman 

Department oF Arts and Cultural Studies, Univer- 
sity oF Copenhagen 



I. Ublconnp Cultures: Hyperbolic Vision, Factual Developnnents 



This is a journal issue invested in remarking more than once upon the undecidability hover- 
ing today around our getting into contact with 'ubiquity' or 'pervasiveness' as a potential to 
be further actualized in the fields of human-computer interaction (HCI), interaction design, 
and the cultural life worlds of information societies more generally. It could well be that you 
have not yet heard of ubiquitous or pervasive computing, or that you have heard of these 
but still remain in doubt whether there actually is or will be such a thing, in interaction de- 
signs or elsewhere. It could also very well be the case, however, that you both know a great 
deal about this as a rather momentous shift, qua a third wave in computing and associated 
disciplines, and find yourself engaging with it all around you in your practical life: at work, 
at home, in leisure activities and games, in the media art at the museum, or in the everyday 
culture of the public sphere. Affirming this undecidability is a necessity - since both of these 
alternatives are currently at stake, and since 'ubiquity' and ubicomp remain potentialities of 
whose actualization we are not yet sure. This undecidability may be a matter of the explicit 
articulation of principal ideas. At the same time, it may concern the concrete lines of devel- 
opment and research that make of this so many hands-on facts inherent in the interactions 
in our contemporary life worlds. In other words, the focus and special merit of this issue is 
not least to enter into the set of questions surrounding the notion of 'interaction designs 
for ubicomp cultures' - as something partaking of that which Michel Foucault would have 
called 'a history of the present.' This issue engages with an altogether contemporary field of 
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research in order to make a difference that makes a difference while the cultural and technical 
developments at stake are still undecidable, multiple, and emergent - at a fast pace, too. 



Even now, 20 years after Mark Weiser's initial coinage of 'ubiquitous computing' as a term 
(1988), and 15 years or so after his seminal papers (Weiser, 1991, 1993, 1994, 1995, 1996, 
1997, 1999; Weiser and Brown; Weiser, Gold and Brown), there is not yet a general consensus 
as regards the definition of 'ubicomp.' However, perhaps one might, as do the contributors 
here, proceed on the assumption that the field of inquiry involves in particular a third epoch of 
computing (after the mainframe and the personal computer), one preoccupied with the ques- 
tion whether and how computing is, should be, or could be moving on from existing primarily 
as distinctly recognizable units so as to be multiplicitously and pervasively integrated into our 
living and working environments and perhaps altogether invisibly embedded in our life world 
and life form. In that case, a working definition of 'ubiquitous computing' would be a socio- 
cultural and technical thrust to integrate and/or embed computing pervasively, to have infor- 
mation processing thoroughly integrated with or embedded into everyday objects and activi- 
ties, including those pertaining to human bodies and their bodily parts. Thus, if somebody is 
living with ubiquitous computing, it does not concern or only marginally concerns engaging 
consciously with a single device or application for some definite purpose. Rather, it concerns 
engaging with multiple computational devices and systems simultaneously during more or 
less ordinary activities, without necessarily being aware of doing so. This is also to say that 
the models and practical implementations of ubiquitous computing investigated here largely 
adhere to something like Weiser's vision of a myriad of small, inexpensive, robust, networked 
information processing devices, perhaps mobile but certainly distributed at all scales through- 
out everyday life and culture, most often turned towards distinctly mundane, commonsensi- 
cal, and commonplace ends. When this sort of extensive distribution is referred to as a de- 
velopment of 'out-of-the-box computing,' it is due not least to Weiser's suggestive notion of 
a process of drawing computers out of their electronic shells so as to have the 'virtuality' of 
computer-readable data and 'all the different ways in which it can be altered, processed and 
analyzed... brought into the physical world' (Weiser, 1991: 98). 



During these last two decades, we have had a number of conferences concerning the tech- 
nics of ubiquitous and pervasive computing, many of which have resulted in the publication of 
conference proceedings, and these conferences now continue on a regular basis (Dourish and 
Friday; Krumm; Indulska; LaMarca, Langheinrich and Truong). Likewise, the interested reader 
will today be able to find at least a dozen books treating of quite a few of the pertinent issues 
in the hardware-engineering and the software- or middleware-development for ubicomp (Adel- 
stein; Cook and Das; Szymanski and Yener). In addition, the first more substantial and useful 
anthologies in computer science studies have now appeared (Symonds, 2010, 2011). Nonethe- 
less, a major part of the technical research issues are far from resolved. Device components, 
networks, as well as the different layers of protocols and applications remain in multiple 
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strands of basic development rather than already being involved in undertaking a broader 
and higher level abstraction from a shared consensus or standard. Non-resolved computer 
science issues that are still common to most research projects at work on ubiquitous and 
pervasive computing include, among other things: the sensation and collection of meaning- 
ful data on 'human' activities; building models for real-world 'human' activity; application of 
software agent technology, many of them only loosely integrated; appropriate unobtrusive 
interfaces; taking the right kind of care of security, privacy, ownership, and trust; apprais- 
ing 'human' factors and social impact; implementing, maintaining, and developing dynamic 
communications networks; managing the scales and heterogeneities of ad hoc networks in 
non-hierarchical ways; modeling collective failure modes; appropriate consideration and de- 
sign of energy consumption when many of the systems depend on batteries (Steventon and 
Wright: 12; Crowcroft). 



In the human and social sciences, moreover, the field of research is characterized by a very 
noticeable delay in the development of cultural theoretical, sociological, psychological, and 
aesthetic approaches to ubicomp and its implications for our form of life. Some conferences 
have now been held, however, in Yokohama, New York, Weimar, London, and Copenhagen, 
mostly by culture and art organizations, a few by universities. In addition, an initial set of in- 
teresting book-length studies have begun to emerge. Malcolm McCullough and Adam Green- 
field's individual studies provide a quite detailed account of what is at stake culturally and 
architecturally in the emergence of ubiquitous and pervasive computing, while drawing each 
in their own way on a sound, vocal skepticism so as to point towards a first set of critical 
evaluations (McCullough; Greenfield, 2006). The Throughout volume edited by Ulrik Ekman 
presents the first relatively comprehensive anthology engaging in an explicit treatment of 
a considerable subset of the socio-cultural, ethico-political, media-specific, aesthetic, and 
philosophical aspects and implications of the contemporary development of ubiquitous com- 
puting (Ekman). 



Something analogous to this delay is at stake for the disciplines and fields more sharply in 
focus in this journal issue, i.e., interaction design, HCI, CHI, and human factors. In spite of 
the undeniable general rise of interaction design since the early 1990s (after Bill Moggridge 
and Bill Verplank's introduction of the term, and after the emergence of the first correspond- 
ing disciplinary institutes), one cannot but notice a certain slowness and several lacunae in 
the engagement with an ever more extensive actualization of ubicomp cultures and their as- 
sociated technics, software, as well as medial interfaces. 



As recent a book as About Face 3; The Essentials of Interaction Design (Cooper, Reimann 
and Cronin) illustrates with abundant clarity that the very vast majority of practitioners today 
still generally subscribe to a view of this discipline as a matter of defining the behavior of 
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products and systems that users can interact with visually, specifically via a graphical user in- 
terface (GUI) appearing on a screen - and this typically by engaging peripherally via a mouse 
and a keyboard belonging to a very recognizable personal computer with Internet access. 
Along that path, the keystones of work on interaction design notably include a distinct (per- 
sonal) computer, a screen, a GUI, and WWW networking. If questions of 'ubiquity' and 'perva- 
siveness' present themselves at all, these appear matters of marginal import. 



However, one would also want to argue that this state of affairs is to be approached as a 
question of a hegemonic and predominant disciplinary paradigm for the second wave of com- 
puting (personal computing and distinct PCs overtly demanding our attention) that is currently 
very much in the process of being revised. For in parallel, and simultaneously, you also find, 
for example. Jay David Bolter and Diane Gromala participating in the SIGGRAPH 2000 digital 
art show only to write a book that argues in favor of establishing as the goal of interaction 
design the appropriate rhythmic oscillation between transparency and reflection, and between 
the invisible computer and the visible computer (Bolter and Gromala: 2-6). Notably, Bolter and 
Gromala wish to see this oscillation unfolding as a matter of a contextual design that is in- 
formed by the new paradigm of relations between the virtual and physical. That is rather pre- 
cisely what one finds in the current developments of augmentation (AR and AV), mixed reality 
(MR) (Milgram and Kishino), and a kind of ubicomp that includes not least an embodied expe- 
rience of decidedly social and cultural settings in the real environment (Bolter and Gromala: 
114-40). From this other perspective, more traditional approaches in interaction design could 
well be supplemented not only by such efforts found in the early T-Garden project examined 
by Bolter and Gromala, but also, for example, by the many later interaction designs that have 
been evolving for tangible AR applications and for embodied sociocultural MR interactivity. 



An analogous set of more traditional conventions can be observed at play in the field of HCI, 
whenever computer science and the behavioral sciences are brought to meet in visual design, 
or whenever computers, operating systems, plus programming languages are brought to meet 
social sciences, cognitive psychology, plus linguistics in a relation of computer graphics and 
communication theory. The enormous impact on interaction designs for personal computing 
made by ocularcentrism, transparency, as well as GUI and WIMP paradigms is evidently still 
with us, hegemonically too. Nonetheless, the pursuit of actual developments of ubicomp qua 
an out-of-the-box 'calm' computing (Weiser and Brown, 1996) has also had its effects on HCI 
during the last period. Witness the responses to the remarkable de facto expansion of embed- 
ded computing found in the EEC 'disappearing computer' research initiative (Streitz, Kameas 
and Mavrommati), Donald Norman's work on the invisible computer (Norman, 1998), as well 
as Paul Dourish's still seminal work on a notion of interaction design for ubicomp cultures that 
would draw upon key insights from HCI, an existential phenomenology of the body, and a phe- 
nomenology of the social world (Dourish, 2001). 
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Given such developments, perhaps it is not altogether surprising to find that already the first 
part of the massive work from HCI International edited by Julie Jacko and Andrew Sears, 
deriving from conferences in 2003, came to include a substantial prospective section on 
virtual, mixed, and augmented environments which very evidently involve, at least to some 
extent, innovative post-PC, post-desktop, and post-GUI notions of interactivity and interac- 
tion designs which might draw upon invisible processes of computation (Jacko and Sears: 
1103-1308). Two years later, the ubicomp conference in Tokyo already bespeaks a deep 
engagement with the challenges inherent in developing fast and robust interfaces for ubiqui- 
tous applications operating in context-aware mixed-reality systems (Gajos et al.). Here, shifts 
from interaction to participation become much more explicit, along with a reconfiguration of 
space as partaking of mixed reality, not least as a matter of a mobile, embodied interaction 
rather than a more or less virtually abstract, primarily sedentary, and positionally fixed im- 
mersion. 



Perhaps the year of 2005 can be considered a minor turning point, as Mike Kuniavsky has 
also observed in his study concerning experience design for ubicomp (Kuniavsky), for the 
later editions of the key disciplinary works from HCI International (Jacko, 2007, 2009, 2011; 
Sears and Jacko) bespeak a growing concern with HCI for ubicomp cultures. Witness the ex- 
panded general treatment of multimodal interfaces, adaptive interfaces and agents, mobile 
interaction design, tangible user interfaces, and information-intensive environments - and, 
more specifically, the devotion of book-length studies to issues of ambient, ubiquitous, and 
intelligent interaction (Jacko, 2009). 



Evidently, the kinds and degrees of actualization of interaction designs for ubicomp cultures 
are not least affected by the ideas and forces which shape computing during the final years 
of the 1990s and the first decade of the new millennium. Interaction designs for ubicomp 
cultures become more than a potential and a much more pressing factual concern in tandem 
with: decreasing general hardware costs, reduction in power requirements, implementa- 
tion of ubiquitous ad hoc networking (including high speed and/or wireless LAN and WAN), 
increasing development of mobile and distributed computing, widening of the ongoing 
deployment of embedded computation to include networked communications among units, 
deployment of materials for miniaturization and further specialization of sensors and actua- 
tors, increased portability of computational devices, thin and large new display technologies, 
pursuit of high-bandwidth interaction and innovative multimodal input techniques, presen- 
tation of group interfaces on organizational and socio-cultural levels, as well as extensions 
of user-tailorability to include user-innovation in more domains. However, these interaction 
designs concern a co-evolution of culture and technics. Thus, such interaction designs de- 
velop in tandem with the broad cultural integration in everyday practices of digital identity 
systems, social media and web 2.0, mobile communications, GPS and locative media, things 
that think (RFIDs), tagging of the life world, information intensive environments, context- 
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aware installations, responsive architectures and smart homes, security systems, surveillance, 
and more. 



Since projects are currently unfolding along such multiple lines of innovation, ubicomp cultures 
cannot simply be termed an actual fact or process, but necessarily retain a considerable po- 
tential dimension, still undecidable and in emergence. Nonetheless, already in 2005 this third 
wave in computing has established enough of a cultural presence empirically as well as in 
research and development that educational textbooks for designers operating with and along- 
side the computation of the 2P* century take as a point of departure a number of the traits 
and challenges of ubicomp. 



In this special issue of Fibreculture and in the contributors' research articles it is taken for 
granted that the relation between the potential of ubicomp culture and its actualization is to- 
day still undecided but also a matter of a very dynamic and energetic set of ongoing research 
projects and concrete technocultural developments which call for quite some descriptive, ana- 
lytical, and critically evaluative efforts. The articles in this issue thus proceed after having ac- 
knowledged what one might wish to call a certain ambiguity. First, the contributors acknowl- 
edge that a fully developed, robust, pervasively distributed, relatively smart, context-aware, 
and innovatively ad-hoc networked ubiquitous computing has yet to emerge as a cultural and 
technical fact, whether in an invisibly embedded infrastructural variant or an overtly attention- 
getting personalized portable one. [1] Secondly, the contributors affirm, at the same time, 
that the actual technical developments as well as our modes of interactivity (socio-culturally 
medial and communicational, psychological, and aesthetic) have already changed enough to 
warrant the recognition that in a number of ways we are living in a ubicomp epoch and world. 



However, this ambiguity is not simply acknowledged in a neutral fashion. The contributions 
can be read as a constellation of statements to the effect that at this point in time the dis- 
courses and practices relating to interaction designs for 'ubiquity' call for a first set of critical 
distinctions. Preferably, one would distinguish between 'ubiquitous computing,' as a historical- 
ly specific term denoting certain actual socio-cultural and technological developments during 
the last two decades relating in the main to computer science, software engineering, interac- 
tion design, media studies, media art, and their supports in the human and social sciences, 
and, on the other hand, the more metaphorically slanted terms 'ubiquity' and 'pervasiveness' 
which appear consistently as idealities, not least in the hyperbolic form of philosophical tropes 
with metaphysical and/or ontological remainders that display quite some traditional capacity 
to survive. 
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Efforts to begin delimiting the latter are evident, for example, in Anders Michelsen's insist- 
ence - in 'Pervasive Computing and Prosopopoietic Modeling' - that we must recognize a 
certain 'cybernetic metaphoricity' within existing claims, more or less explicit, that comput- 
ing and its mediations can 'pervade' realms of the real so as to really be or become 'ubiq- 
uitous.' Michelsen suggests that we first undertake a more detailed rehistorization to see 
how this metaphoricity displays intrinsic relations to the heritage of cybernetics and systems 
theory from the mid-20* century onward. On that background Michelsen suggests that 
we address 'ubiquitous computing' as partaking of more than half a century of computa- 
tional imagination, specifically engaged in a reworking of what one might call, echoing the 
thoughts of Herbert Simon and Ezio Manzini, 'an imaginary of the artificial' (Simon; Manzini). 
At this altogether general level, interaction designs for ubiquity would then best be ap- 
proached as a matter of a creative human articulation, Michelsen argues, in which one may 
distinguish between a novel form of our computational modeling of the real via the artificial 
(approximating the world as design) and, on the other hand, the more radical address of the 
artificial as being computational per se (approximating a radical computational imagination 
or a process of artificialization). 



Whether in cultural theoretical or technical discourses, the terms of 'ubiquity,' 'pervasive- 
ness,' and 'ambience' come silently freighted with a notion of totalizing universality or even 
certain ontological and metaphysical remainders (altogether abstract idealizations and/ 
or excessively essential or substantial extensions). Both the editors and the authors con- 
tributing to this special issue approach this as a call for ongoing deconstruction and recon- 
struction, not least in the sense that remainders and implications of onto-theological and 
sovereign ideological notions must be questioned reasonably so as to be put under critical 
erasure in one or more ways. The articles thus include an implicit orientation towards rather 
unconditional critique of the idea that ubicomp is, should, or could be 'ubiquitous,' that 
pervasive computing is, should, or could be 'pervasive,' that ambience is, should, or could be 
'all around,' or that the discourses, practices, and inventions involved extend, penetrate, and 
invade 'throughout,' or are always already at stake all over as an omnipresence. Instead, one 
would like to put the emphasis on the multiple actual ways in which interaction designs for 
'ubiquity' partake of infinite finitude. Perhaps a reminder of our myriad on-off relations of 
interactivity with and within mobile cultures is one of the easiest and best ways to illustrate 
that the problematics of cultures of ubiquitous information exist not as a totality or infinity 
but rather as so many matters of immanent complexity. In this actual but still emergent third 
wave of computing, its mobile devices and co-developing cultural practices might be one of 
the best foci. This because all our everyday engagements with mobile phones, handhelds, 
and small tech make felt a culture of ubiquitous information qua the dynamics and ener- 
gies of ad hoc network theories and practices - live as organized inorganicity, inorganically 
and organically live. Mobile computational entities and their cultural enfoldment are such 
good foci because they make felt the ways in which complexity arises from a vast number 
of distinguishable relational regimes and their associated state spaces, promising a defined 
system of interactivity for 'ubiquity' (to come). 
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II. Interaction Designs: 

Technical and Human Co-Developments in Question 



In the main, then, the contributions to this issue proceed on the assumption that an uneven 
and uncertain but nevertheless quite recognizable developmental trend is already a fact, 
something which involves ubicomp cultures as actual harbingers of a third wave of comput- 
ing as well as a gradual factual articulation of a paradigm for those many interaction designs 
now arriving which are in a number of respects different from those we already know very well 
and continue to live with, i.e., those pertaining to engagements with a predominantly personal 
kind of desktop computers, their GUIs, windows, icons, menus, and pointing devices (WIMP). 
[2] In the new millennium, and notably after 2005, as a citizen of more or less massively 
networked information societies, one has already been interacting enough beyond command- 
lines, menus, desktops, and GUIs to have realized that another set of models is operative, and 
that there is at this point an obvious need to pursue analyses and critical evaluations of these 
models. Along with a great many innovative artistic explorations in new media art installa- 
tions, you may witness this development, as do the contributors here, on a larger socio-cultur- 
al scale. 



One meets it, for example, via the signals and vibrations from mobile phones, when drawing 
upon the services afforded by the massive development of wireless devices and overlapping 
networks (including WPAN, WLAN, and WMAN or WiMax), when engaging with the tangi- 
ble interfaces and auditory culture of MPS players (iPods), when you partake of the reading 
anywhere of e-books on your networked iPad or Kindle, via the RFID tagging of your library or 
supermarket, when getting upset by the locative and navigational practices unfolding along 
with GPS, via the entry into your workplace of interactive whiteboards, during your increasing- 
ly frequent architectural encounter with smart buildings and environments, via the experience 
of augmented museum tours, and not least in the rapid and extensive widening of effects 
from the technics of surveillance (CCTV, webcams, security video cameras, sensor networks, 
profiling and behavioral recognition, as well as biometrics and digital identity systems). Along 
with the rise of wearable computing, prosthetics, and implants, perhaps one can best appraise 
the entire contour of this actual development, i.e., its serpent-like and environmental recoiling 
in and around the life of the anthropos as such, by considering the integrative and embed- 
ding efforts characteristic of pervasive healthcare, in the hospital, of course, but certainly also 
in the milieu of city and the context of the home (Orwat, Graefe and Faulwasser; 'Centre for 
Pervasive Healthcare'; 'Center of Excellence for Ubiquitous Systems'). 



Accordingly, in such a work as Designing Interactive Systems, appearing in the year 2005 
(Benyon, Turner and Turner), it is taken for granted that interactive systems design is now 



8 - Issue 19 fibreculturejournal.org 



Ulrik Ekman 



primarily a question of contextual design, much as Bolter and Gromala, Dourish, and oth- 
ers have begun to suggest. This is also to say that perhaps context-awareness has become 
one of the major traits of interactive ubicomp systems. Now HCI also concerns information 
spaces whose architecture, design, navigation, and agents must be considered, as when 
undertaking interaction design for the dynamics of computer-supported cooperative working 
(CSCW) or social mingling in an airport or on a public square. Moreover, then, such interac- 
tion designs for ubicomp cultures not only draw, as did earlier HCI, on cognitive psychology 
to understand and support single-user interaction with a computer. They also extend this 
psychologically and behavioristically to begin encompassing multi-user interaction involv- 
ing many computational units in physical and socio-cultural settings for AR as well as virtual 
environments with AV. Thus, one must assume interaction design to include not only a single 
individual's abstract, virtual, and sedentary interactions with a personal computer, but also 
embodied, distributed, locative, and mobile cognition on the side of many human users and 
systems both. 



This is also to remark that the context-awareness at stake in interaction designs for ubi- 
comp cultures slowly tends towards operating with the entire span of mixed realities, now 
notably including the real environment and interactants' embodiment. In that respect, the 
2000s might well be distinct from the 1990s fascination with a transcendent virtuality, as 
Lev Manovich has also observed, in that the period after the millennial turn turns out to be 
about the physical - physical space filled with electronic and visual information. Along with 
the pursuit of ubicomp augmentations and mixes, computation and network technics active- 
ly enter our real physical spaces, replacing the well-known cognitive model of a VR interact- 
ant traveling in a virtual space by an embodied model of a person checking her email or 
making a phone call using her PDA / cell phone combo while in the airport, the street, a car, 
or in any other actually existing space (Manovich). 



The arrival of interaction designs for more fully mixed realities is still quite delayed and 
deferred in several ways, and our notions of 'mixed reality' remain varied. [3] When MR is 
pursued, the 'mix' at stake is hardly ever addressed explicitly but understood as a matter of 
augmentation, which thus tends to be treated as medial overlays or spatiotemporal super- 
impositions rather than 'mixes' involving conversions of the digital and the analog. Likewise, 
when augmentation is pursued in current interaction designs, it is likely to be understood as 
'improvement' just as augmented reality seems to have the upper hand (perhaps due to the 
wish to counter more than a decade of privileging of VR). Currently, one must observe a ten- 
dential parenthesizing not only of the exploration of interactions with a virtual environment 
including augmented virtuality, but also of any critical engagements with the notion of more 
or less teleological 'progression' and 'improvement' assumed to be at play. 
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Context-awareness can only be termed a chief goal for ubiquitous and pervasive computing 
and projects within both do display significant efforts to meet this. Developers, interaction 
designers, and users of ubicomp have so far tended to grant a certain primacy to the physi- 
cal environment and a rather empirical notion of 'context,' as in almost all locative projects 
drawing upon GPS, for example. This kind of focus still in the main ignores or parenthesizes 
the human users and their context-awareness. It bespeaks a keeping to key decisions made 
in early cybernetics (Shannon and Weaver) as regards marginalizing or altogether excluding 
the exploration of 'context' as a matter of meaning, i.e., in general any approaches stemming 
from linguistics, discourse analysis, hermeneutics, semantics, semiotics and, quite specifically, 
the early alternative cybernetic theories trying to work integrally with aspects of this (MacKay; 
Ruyer). This bracketing is, however, not unilateral and also seems to be changing over time 
as research interests now begin to accrue to the role in ubicomp of human awareness, mean- 
ingful embodied action and response, as well as language. Hence it is of some interest to 
keep on tracing this development. Nonetheless, although context-aware ubicomp systems are 
already quite advanced as regards human user profiling on biophysical, perceptual, and some 
limited habitual action planes, it is still a challenging and open issue, to put it mildly, how to 
have context-aware systems try to make relatively well-informed or even intelligent assump- 
tions about users' current situations. We still await, then, an in actu and proactive demonstra- 
tion of cybernetic systems that incorporate and interact with well-interpreted human intention, 
individually and socially. It seems clear today that a Pandora's box for cybernetics has been 
reopened once more with this call for establishing and engaging with meaningful contextual 
relations. That is, an approximation to meeting human context-awareness still remains a for- 
midable challenge (for technics, interaction designs, and new media), and the hindrances and 
misunderstandings remain many as soon as a move is made from a systemic awareness of 
physical location and movement in positional terms to an awareness of human affect, pas- 
sion, intention, action, emotion, perception, or semantics. [4] 



Even if interaction designs involving 'mixed realities' and 'context-awareness' today still ex- 
plore the polysemia of both, it is quite evident that these two notions have become key traits 
because they represent attempts to actualize a central heritage from Weiser's early vision of 
ubiquitous computing. For the current efforts to develop and engage with mixed reality in- 
teraction designs include not least a response to Weiser's call for a remarkable expansion or 
widening of the field of research. Ubicomp should become an out-of-the-box computing, i.e., 
it should draw 'computers out of their electronic shells' and into the physical world so as to 
'become part of the environment,' or become integrated 'seamlessly into the world.' This move 
towards 'embodied virtuality' cannot but involve a notion of mixed reality, just as it remains 
almost diametrically opposed to the virtual reality paradigm of trying to make a world inside 
the computer (Weiser, 1991: 94-96). Likewise, the very intense pursuit of interaction designs 
involving context-awareness echoes part of the heritage from Weiser and his colleagues at 
Xerox Pare. In this case, the disciplines of interaction design and HCI try to respond to the 
call for a human-centered and 'calm' computing. This would entail development of interac- 
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tion designs for computational processes that have become an integral, invisible part of the 
way people live their lives, processes that may 'calmly' vanish into the background because 
having taken into account the natural human environment (Weiser, 1991; Weiser and Brown, 
1996). As revealed by Weiser's explicit aims to have computers free us to use them without 
thinking and so focus beyond them on new goals, to have computing free us to be more fully 
human, the pursuit of both these parts of the heritage either assumes or demands a very 
thorough systemic grasp and knowledge of the human being, its notion of reality and its 
practical mode of being aware of or attentive to its situation and context. It is relatively safe 
to say that neither computer science nor interaction design and HCI has so far been able to 
live up to this, leaving us with a great many interesting de facto research projects over the 
last two decades, a set of unresolved and perhaps unfeasible technical aims as well as an 
equally important set of critical discussions respecting not only the cultural desirability of 
these aims but also what might present alternatives to be considered. 



In principle, the vision for ubicomp signals a complete out-of-the-box expansion that would 
entail the development of interaction designs for all human interactivity in mixed realities 
presumably present anywhere and at all times. In fact, this is neither technically feasible 
nor culturally desirable, and thus one would perhaps do better to put in parenthesis the 
hyperbole - so as to observe the need for further concrete delimitations. Generally speak- 
ing, mixed ubicomp realities are rather few and far between, not least because the massive 
infrastructural demands for high-speed wired net connections as well as many more wired 
or wireless nets of sensors and actuators have not been met. Interaction designs for these 
mixed realities so far tend to appear in rather strictly delimited or specialized areas such as 
technical research departments, pervasive healthcare, singular architectural projects, office 
environments for large corporations, large stores or archival institutions, and new media art 
installations. Interesting and notably exceptions would be the much broader technocultural 
developments associated with our interactions with mobile communication and ICT (Gog- 
gin and Hjorth; Ito, Okabe and Matsuda; Jacko, 2011; Webb), security and surveillance after 
9/11 (Frohne et al.; Haggerty and Samatas; Tennenhouse; Want and Pering), and digital 
identity systems (Wren and Reynolds; Michelis et al.; Waller and Johnston). While a great 
many researchers acknowledge the general trend towards an expansion or widening of the 
research field and its practical aims, we are still missing any first overview of the concrete 
areas of interactivity being addressed, not to mention their relationships and the priority 
granted to each. 



With respect to the pursuit of context-awareness, we are much further in certain actual 
developments of this, but we do not entirely agree on how and whether this can or should 
be human-centered, whether it can or should be intelligent, proactive, and self-adaptive. In 
spite of the work by Beigl, Dey, Dourish, Gellersen, Loke, Schilit, and several others, we do 
not even have one generally accepted definition of this notion, not least due to difficulties 
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as regards answering the question of contextual interpretation and semantics (Dourish, 2001; 
Dourish, 2004; Schmidt, Beigl and Gellersen; Loke; Schilit, Adams and Want; Dey). It is still far 
from easy, even now in the day and age of a semantic web on the rise, to see how meaning 
and interpretation are to be handled so as to make a relatively smooth translation from 'infor- 
mation' in computer science to the kind of semiotics and hermeneutics most often taken for 
granted in cultural studies and the human or social sciences more generally. 



It remains an issue of quite some debate, in the articles in this issue as well as more broadly 
in the research community, whether or not we are to pursue mixed reality out-of-the-box com- 
puting and context-awareness as part and parcel of the kind of human-centered computation- 
al integration and embeddedness at stake in Weiser's original vision for a 'calm' computing. 
First, as Manovich also remarks in his contribution to this journal issue, the notion of an alto- 
gether invisible computing and an equally imperceptible interaction design is now to a large 
extent discredited or problematized, in spite of work on this by Donald Norman as well as in 
the EEC disappearing computer initiative (Norman, 1998; Streitz, Kameas and Mavrommati). 
According to Manovich's argument in 'Interaction as a Designed Experience,' the shift during 
the 1995 to 2008 period consists in abandoning the ideal of invisibility in favor of treating in- 
teraction explicitly as an event while designing aesthetic interfaces for a carefully orchestrated 
or dramatized experience. While embedded computing will continue to be on the rise for quite 
some time, along with infrastructural projects (in Europe not least), [5] strict invisibility and hu- 
man unawareness also remain problematic as aims and values for technical, ethico-political, 
and mundane practical reasons (e.g., situations of repair, infringements of rights, or everyday 
workability as well as minimal recognizability of interaction designs). 



Then, one would like to remark that this is also a rather slanted understanding of the vision for 
ubicomp, since Weiser's 'calm' computing only here and there could be read as a call for strict 
invisibility and complete embeddedness. In their more interesting and convincing passages, 
Weiser's texts first of all remarked that not all computing or interaction had to be calm, just 
as 'calmness' is here to be understood as referring to a varied and dynamic engagement with 
human economies of attention, oscillating between center and periphery. Interaction designs 
would thus be at work on a technology that may move easily from the periphery of human 
attention to the center, and back. When interaction designs become peripheral, they would 
arguably permit interactants to attune to many more things by drawing upon the large part of 
the brain devoted to peripheral and sensory processing - i.e., by informing without overload- 
ing. Centering or recentering something from peripheral interactivity would then permit both a 
more focused awareness and a series of interactions aimed at increased control or correction 
(Weiser and Brown, 1995). 
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Read in this way, interaction designs for ubicomp cultures would not be post-desktop, post- 
GUI, post-WIMP in any revolutionary sense, but would leave most of existing efforts in HCI 
intact and useful, only to aim at a certain supplement in the form of an expanded and more 
finely differentiated treatment of kinds and degrees of situated attention. Still, this is by 
no means a simple matter, witness the fact that we do still not have any one agreed upon 
standard for interaction designs for ubicomp cultures. No little part of the difficulty stems 
from the extensive implications of this vision for interaction design. For, as Weiser was well 
aware, interactivity with a 'calm' dynamics calls for interfaces characterized by something 
akin to a visual invariance (J. J. Gibson), a tacit dimension (Michael Polanyi), a vague and 
unobtrusive peripheral reach (John Seely Brown), a heuristic compiling (Herbert Simon), a 
horizontality (Hans-Georg Gadamer), or a being ready-to-hand (Martin Heidegger). In other 
words, what is at stake here involves interaction designs endowed with a practical and sen- 
sate intuition, i.e., designs for situated and lived experience that not least grant a new and 
major privilege to the more passive and perpetually open dimensions of human sensation, 
and hence to the proximate and affective dimensions of our embodied activity and form of 
life. 



One line of response to this consists in pursuing a more nuanced and balanced understand- 
ing of Weiser's vision of an embedded and calm ubicomp - something making itself felt 
today in the area of interaction design research and development. For example, Vivienne 
Waller and Robert Johnston draw upon Heideggerian notions of availability to argue that 
ubicomp will support our everyday activities only when designed so as to be physically and 
cognitively available both - not altogether embedded, nor invisible (Waller and Johnston). 
Christopher Wren and Carson Reynolds refer to Ishii's important work on tangible user inter- 
faces and describe specific ways in which minimalism - not invisibility - in ubicomp interface 
design will allow computational augmentations to coexist with physical artifacts and the 
task behaviors surrounding them (Wren and Reynolds; Ishii, 1997, 2008). Likewise, Hyunjung 
Kim and Woohun Lee present concrete projects for everyday ubicomp objects and practices 
for which physical plasticity and minimalist design may provide unobtrusive interfaces that 
shift between invisible and visible states (the concealed interface appears when put into use, 
and disappears after use) (Kim and Lee). 



Benyon and others emphasize that the cognitive and psychological foundations for interac- 
tion design will continue to draw to some extent on transparent presence, a consciously 
clear vision, and an explicitly focused practical understanding. However, another privilege 
is now to be granted to the role of largely silent memory traces, subtly peripheral attention, 
and an inclusive approach to the process of making of mistakes and corrections. Likewise, 
with the unfolding of multimodal interaction designs hearing and haptics would become 
vastly more important, as would the roles of affectivity and degrees of being pleased or 
unsettled (Benyon, Turner and Turner: 352-447). Perhaps one of the most obvious changes 



fibreculturejournal.org Issue 19- 13 



Editorial: FCJ-19 Ubiquity 



heralded here is the relative and gradual downplaying of the digital visual culture of the 
screen (not the outright disappearance proclaimed by Manovich as the historical outcome of 
the arrival of embodied virtuality after the three main phases of the classic, the dynamic, and 
the real-time screen, or by Michelis), since the more implicit modalities of haptic, auditory, 
and ambient interaction will be granted another priority (Michelis et al.). Albrecht Schmidt and 
others point out that the traditional focus in HCI on interfaces for explicit use (visible GUIs and 
widgets on screens) remains but a small part of the expanded design space for interactive sys- 
tems from a ubicomp perspective - which would include not only tangible and physical user 
interfaces that go across explicit and implicit modes of interaction but also (and not least) 
implicit modes of interaction via movement, posture, facial expression, gesture, direct manipu- 
lation, GUIs, command lines, and speech. An examination of this expanded design space will 
show that embeddedness or invisibility does not necessarily determine how a system is used. 
Something perfectly embedded and hidden can still be used explicitly - and non-embedded 
or visible technology which is used often can very well be used implicitly and without the user 
being aware of it. This goes quite some way towards pointing out the limited use of adopting 
any simple concepts of physical embeddedness and invisible computing in this field - since 
the question is not least one of degrees of experienced invisibility and interactions embedded 
into a user's task or more or less habitual situated action (Schmidt, Kranz and Holleis). 



It might be that William Edmondson and Russell Beale sum up a significant part of our cur- 
rent, more nuanced understanding of Weiser's notions of a 'calm' computing when they claim 
that in our environment today we find not one seamless, invisible computational mesh but 
rather a grouping of at least four kinds of ubicomp - many computers; people using comput- 
ers much of the time; embedded computers; 'invisible' computational systems - with the two 
former gradually undermining the two latter and more specialized notions (Edmondson and 
Beale). Insofar as this is correct, we are in a specific sense witnessing a relatively strong but 
gradual disembedding of computers -- as they appear more overtly in our everyday cultural 
practices. This is still only one point of view, though, and in more general terms we are still 
not done discussing how and whether interaction designs for ubicomp cultures are to draw 
upon invisible, visible, peripherally attention-getting, or peripherally vague processes of com- 
putation. At best, we have a first general understanding that a mix of all four of these will be 
called upon for the interactive parts of ubicomp to function as expanded out-of-the-box com- 
putation in and as our life world. Robert Jacob and others are no doubt right when rejecting 
the idea that the numerous actual moves towards post-WIMP interaction styles (such as VR, 
MR, AR, TUI, handheld, mobile, lightweight, tacit, or passive) during the last two decades are 
not to be seen as disparate innovations proceeding on unrelated fronts, but rather as sharing 
important commonalities. The ongoing realizations of ubicomp are at one in reality-based in- 
teraction, according to Jacob et al, and build on users' pre-existing knowledge of the everyday 
non-digital world to a much greater extent than before, notably by drawing upon users' under- 
standing of naive physics, their own bodies, the surrounding environment, and other people 
(Jacob et al.). 



14 - Issue 19 fibreculturejournal.org 



Ulrik Ekman 



In more ways than one, this echoes Simon Penny's claim in his contribution to this issue 
that the kinds of interactivity we encounter today in the media art projects engaging with 
ubiquitous computing partake of a general historical development towards a performative 
ontology and aesthetics of interactivity. Penny provides a much needed and quite detailed 
historical account of 60 years of intersections between technological development and 
artistic experimentation, including a critical reconsideration of the lack of non-instrumental 
theorizations of interactivity. In this rehistorization. Penny is generally at pains to demon- 
strate that throughout the period of pioneering interactive art up to this point in time one 
can, and should, trace the moments of a development moving towards a synthesis of perfor- 
mance theory and neurocognitive studies which forms the base of that performative ontol- 
ogy on which current aesthetics of interaction and interaction design is to be constructed. 
Within this broader historical tapestry interaction designs for ubicomp cultures are woven as 
relatively mature latecomers - in the sense that their integrations of materiality and compu- 
tation, technology and culture, or control and action bypass any dualist approach. Penny's 
double claim is thus first that in the current epoch of ubiquitous computation, the universe of 
live data is increasingly anchored in physical and social contexts via a variety of digital enti- 
ties and, secondly, that the technics, the techno-social strutures, as well as the modalities of 
interaction we now see permitting of such (re)unions were essentially worked out and proto- 
types in media art research during the past quarter century. 



However, having acknowledged all of this, one must also recognize and affirm another 
main line of response, for enough time has passed since Weiser's untimely death that more 
explicitly critical arguments respecting his ideas begin to appear in tandem with that ongo- 
ing realization of the ubicomp vision which is in a number of respects different from what 
the researchers at Xerox Pare imagined early on. Yvonne Rogers suggests that we now step 
aside from the intractable computer science problems and the considerable ethical problems 
lurking in Weiser's vision in order to explore alternatives to 'calmness,' such as designing 
for explicitly engaging user experiences, designing so as to extend creatively, excitedly, and 
constructively what people currently do, or designing in order to support personal, cognitive, 
and social processes of habit-changing, problem-solving, creativity, learning or performing a 
skill (Rogers). 



In 'Affective Experience in Interactive Environments' in this issue, Jonas Fritsch assumes, to 
begin with, that Rogers is right in proposing as an agenda for working with ubicomp that 
interaction designs had better focus on the creation of engaging user experiences, just as 
objects or environments with embedded interactive technics now should tend towards be- 
coming remarkable lifestyle objects or places with which we engage in identity formations. 
In that case, designing for living with ubicomp must imply considerations of the experiental 
qualities brought into play to form our concrete experiences of interactive objects and en- 
vironments. Accordingly, Fritsch singles out the question of affectivity as the primary focus 
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of interaction designs for ubicomp cultures. For interaction design now needs to develop an 
understanding as well as means to describe how interactive environments and technics may 
shape our situated affective experiences. Via an exemplary case study and analysis of the 
experiental field of the interactive installation City Voices, Fritsch seeks to confirm and flesh 
out his hypothesis that Brian Massumi's concept of affect is particularly useful for investigat- 
ing the ways in which ubicomp environments engage their human users affectively by creat- 
ing fields of experience. Arguably, this concept will permit the needed kind of focus on such 
affective tonalities and such relational events as might emerge through interactions in these 
environments. 



Bo Kampmann Walther's contribution to this issue, 'Reflections on the Philosophy of Pervasive 
Gaming,' focuses on the phenomenon of pervasive gaming still on the rise, not least in urban 
settings, as attested to by SuperFly, by the Swedish game company It's Alive Mobile, for ex- 
ample, or by projects such as Can You See Me Now? and Uncle Roy All Around You by the UK 
performance group Blast Theory. Thus Walther seems to agree with both Rogers and Fritsch 
that experiences of play, creativity, problem solving, learning, and skillful performance are at 
the heart of living with ubicomp cultures and their interaction designs. Walther's text not only 
contributes to discussions of new (post-GUI, post-screen) pervasive interfaces but also seeks 
to legitimize and flesh out significant parts of 'pervasive ludology' as the field of pervasive 
gaming theory. Generally speaking, Walther's inquiry concerns a certain meta-ludic architec- 
ture of pervasive games, or that which constitutes the playability and epistemology of post- 
screen pervasive games. This entails a reexamination of the notion of 'gameplay' in terms of 
rules, tactics, exploration, and level-orientation as well as acts of playing vs. acts of gaming. 
Quite specifically, Walther seeks to reflect on Deleuze's claim that the virtual is not opposed 
to the real but must be understood as the condition for actuality. Here Walther argues that the 
Deleuzian concept of virtuality is precisely what can be used to explain the relational structure 
of play that connects and yet remains irreducible to gaming. 



Malcolm McCullough's work on the digital ground for architecture, place, and out-of-the-box 
computing draws more explicitly and affirmatively upon the early visions for ubiquitous and 
pervasive computing than do Fritsch and Walther. However, McCullough's extended and quite 
detailed account of how embodied virtuality is to be inscribed into the social and environ- 
mental complexity of the existing physical environment nonetheless insistently gives voice to 
a number critical issues. It is far from certain that such situated technology will help us man- 
age the protocols, flows, ecologies, and systems that secure valued places. Without drawing 
upon very rich cultural foundations, McCullough argues, it is quite likely that layers of dis- 
trust, information glut, and experiental uniformity will be the result. Acknowledging the rise of 
interaction design, in this field too, McCullough agrees that digital systems which are carried, 
worn, and embedded into physical situations can alter how people interact. Nonetheless, it is 
uncertain that the new remote, asynchronous, and indirect modes of interaction will come to 
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serve an environmental knowing and our need for 'getting into place,' since software engi- 
neers have pursued the accumulation rather than the integration of technical features, since 
interface designers have so far focused on first-time usability at the expense of more satis- 
fying long-term practices, and since psychologists, ethnographers, architects, and cultural 
theorists barely understand the consequences of all this mediation (McCullough: xiii-xiv). In 
his article for this issue, McCullough further expands on his critique concerning the lacunae 
still left in current research and development of ubicomp cultures and their interaction de- 
signs. The text 'Toward Environmental Criticism' can perhaps best be read as an urgent call 
to have information history and environmental history merge enough so that a phrase such 
as 'environmental history of information' will yield a good deal more than null in our search 
engines (as opposed to now). McCullough rightly observes that the actualization of ubicomp 
mediation and interfaces has already blurred separations of environment and information, 
environment and nature, environment and culture - without the accompaniment of any criti- 
cal consideration of the taggings, annotations, operations, linkings, recordings, fillings, and 
pollutions at stake in the upsurge of information intensive environments. One might well 
contend, then, that an environmental criticism is lacking that would not least address the 
physical embedding of certain environmental sensibilities outside the efficiency of compu- 
tational tasks. Here McCullough would like to have us reconsider the ways in which ambi- 
ent and inhabitable information advances or obstructs larger changes in worldview, and the 
ways in which specific concerns of interaction design imply active responsibility or a neglect 
of that question. 



In this journal issue, S0ren Fold and Christian Ulrik Andersen's contribution, 'The Scripted 
Spaces of Urban Ubiquitous Computing,' continues and further develops such a line of criti- 
cal thinking respecting the implications of ubicomp for our engagements with spaces and 
places. Andersen and Fold argue that contemporary urban environments with ubicomp 
could be approached as so many 'scripted spaces' qua spaces in which ubicomp literally 
has added coded and meaningful scripts to our surroundings by programming smart things, 
architectural equipment, infrastructural devices, and FDAs that all link up wirelessly. As the 
authors demonstrate via a case study of the Swedish city of Lund, such scripted spaces not 
only have a non-visual, coded, and encrypted side but also in some ways go beyond the 
uneasy, anxious, or paranoid experience of computational control structures and transac- 
tions behind the fagade, the surface, or the interface of the city. For the coded and essen- 
tially written character can and should be highlighted as yet another layer in the spectacular 
city, perhaps first as a hieroglyphic sense of concealed meaning or as a sign of signs, but 
then perhaps also as a more or less critical recognition of the ways in which the scriptings 
reorganize perception, teach understandable structures of thought, and tend to internal- 
ize the logic of the programs. According to Fold and Andersen, the distinctly political level 
of scripted spaces remains subject to overt debate as well as internal critique: they can be 
used in unintended ways, be reconfigured, or even hacked. In certain, rather marginal cases, 
the city dweller may be dissatisfied enough by the urban scripts to experience this as a call 
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for action, a call to become 'writerly' in the sense of hacking, rewriting, or creating the city, 
something which will also mean a return of an urban public domain as well as the arrival of a 
paradigm of computing remarkably different from that of the disappearing computer. 



In 'Embedding Response: Self-production as a Model for an Actuated Architecture,' Mette 
Ramsgard Thomsen and Karin Bech are equally concerned with the relation of ubicomp and 
the built environment but this time approached from within architecture as a practice-based 
research discipline. Presenting Breathing Room and Slow Furl, two of their recent installa- 
tions, the authors explore worlds in which the built environment has strata of embedded 
computation and presents a readiness towards the digital and the mediated. Notably, Bech 
and Thomsen seek with these two installations to challenge in at least two ways the prevalent 
approaches to ubicomp and architecture. In the first place, they develop a model of interac- 
tivity that does not focus on engaging a human user in a situation of apparent control but 
rather invests the built environment as a whole with systemic qualities of self-production and 
recursiveness that grant a new privilege to a semi-autonomous pulsation and mobile respon- 
siveness of architectural self-invigoration. Second, Breathing Room and Slow Furl introduce 
dynamic textiles as transitive materials as well as sensing and actuating computational pro- 
cesses embedded so as to have become integral parts of the built environment. This contrib- 
utes to a new architectural language of responsive design that integrates both temporality 
and movement and thus instability as challenges to prevalent material practices of architec- 
ture which grant a privilege to the permanent and the static in their tectonics. In effect, Thom- 
sen and Bech's contribution begin to raise questions respecting what it means for ubicomp 
architecture to become interactive in ways that query traditional spatial conceptualization and 
architectural values such as stability and structure, or what happens to our material practices 
as the ability to sense and actuate become properties of architectures of change. 



Adam Greenfield's easily readable book on 'everyware' not only nods in the direction of the 
becoming actual of ubicomp and pervasive computing (as do McCullough, Fold and Andersen, 
as well as Ramsgard and Bech) but in fact offers an explicit argument to the effect that it is 
an inevitable development (Greenfield, 2006: 91-92). In many ways it is still the best broadly 
accessible treatment from a technically and culturally informed author of ubicomp user ex- 
perience, associated interface modes, and techno-social implications. However, this does not 
preclude quite extensive critical treatments of issues such as information overload, invisibility 
and seamlessness, unpredictable and emergent kinds of interconnections, as well as the risk 
that human agency, judgment, and will progressively become supplanted by compliance with 
external, algorithmically applied standards and norms. Greenfield's frank identification of limit- 
ing and critical factors is refreshing, as when he states that: broad standards for the interoper- 
ability of heterogeneous devices and interfaces do not exist; the network infrastructure de- 
ployed in most places is insufficient to support ubiquitous applications; the appropriate design 
documents and conventions simply do not exist, nor is there a community consciously devoted 
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to the design of ubiquitous systems; there is hardly any awareness on the part of users as 
to the existence of ubicomp systems, nor any agreement as to their value or utility (180). In 
a later, short research article, Greenfield makes his main critical focus on social and ethical 
questions quite clear. If issues such as privacy, security, robustness, surveillance, power, the 
environment, social sorting and control remain, then a set of minimal ethical guidelines is 
one missing resource. In view of this, Greenfield proffers the argument that ubicomp devel- 
opments should: default to harmlessness; be self-disclosing; be conservative of face; and be 
deniable (Greenfield, 2008). 



Moving across both these main lines of response, it becomes increasingly clear that quite 
careful balances will have to be struck between implicit and explicit interaction, almost 
always with the option to enter into the latter, as pointed out by Christina Brodersen and 
Jannie Kristensen, for instance (Brodersen and Kristensen). Their wish to approach inter- 
action designs for ubicomp cultures via a concept of 'negotiation' is also quite interesting 
since it manages to highlight the key problem at stake for Weiser as well as for Rogers and 
many others, i.e., the question concerning the relation between the human and the technical 
potential in a given context or situation of interactivity. Whereas the latter are quite explic- 
itly human-centered in their approaches, Brodersen and Kristensen take another tack which 
promises to continue the long, slow deconstruction elsewhere of the recurring tendency to 
get caught in the opposition between technological determinism and its inverse (ingenious 
human mastery or orchestration). For, defining 'negotiation' as the mediating process relat- 
ing human users and technological possibilities in a given situation, they wish to empha- 
size the mutual translation process taking place in HCI situations, 'taking into account the 
strengths of the computer as a computer and the human as human' (262). 



In spite of the lack here of a definition of either term (human, computer), perhaps this call for 
interactive co-developments or co-evolutions of technics and the human, at the limit of any 
asymmetric relations, provides the most interesting challenge for the mixed reality ubicomp 
cultures and their interaction designs to come. It is perhaps really not simple to say with 
respect to this interactivity whether and how human context-awareness will become techni- 
cally other, whether and how technical context-awareness will become otherwise human. 
Probably, this interactivity will be concerned only very little with further developments of lin- 
ear processing according to pre-established arrays or hierarchical trees of possibilities, just 
as it seems unlikely that it will come to involve thoroughly autonomous, self-adaptive, self- 
reproducing, or even mutant artificial life forms. If interaction designs for ubicomp will not 
be or become a matter of simplicity, however, this is because the mutual translation among 
machinic and human processes of interactivity is emergent - something which presupposes 
complexity. 
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III. Emergent Interaction Designs? 



If ubicomp cultures are not a Utopian imaginary but in some sense 'real,' it is a guiding hy- 
pothesis for the contributions to this journal issue that they constitute a remarkable recon- 
figuration of our mode of living with modern computing and a considerable expansion of the 
fields of interaction design and HCI. In addition, the contributors assume that this realization 
of ubicomp as a cultural and technical fact, already at the stage at which we are today, brings 
with it some quite pressing theoretical and practical concerns for the human, the social, as 
well as the technical sciences. For the actualization of ubicomp cultures must be approached 
as renewed engagements with an emergence arising from cultural and technical complex- 
ity. One key wager here is, then, that we need to revisit notions of complexity and emergence 
both. This is not only because most of us are not yet particularly good at working through with 
these in mind, even though we have to — because only a few of us have the requisite method- 
ologies - since these tend to go across physics, chemistry, biology, network topology, dynamic 
systems theory, and social or cultural theory. A renewed encounter with complexity and emer- 
gence is also a necessity because ubicomp cultures cannot but involve the movements, dy- 
namics, more or less ad hoc relations, and modes of organization or self-organization pertain- 
ing to multiplicities of computational entities as well as humans. 



As a researcher in the human and social sciences concerned with finding or developing an 
adequate set of tools in sociocultural theory this is a tall order. For even after having read, 
for instance, Gilbert Simondon and others on relational ontologies, Gilles Deleuze and others 
on assemblages, Michel Serres on translation without a metalanguage, Humberto Maturana 
and Francisco Varela on self-organization, and various kinds of STS or actor-network theory 
from early Bruno Latour onward, one is still not convinced that one would have the right kind 
of tools to describe, analyze, and critically evaluate 'complexity' and 'emergence' as two key 
implications of ubicomp for our forms or life. What kind of sociocultural theoretical approach 
would you mobilize in the face of an innovative developmental phenomenon like the new tech- 
nical and human movements and relations appearing and disappearing as interactivity in a 
smart building with a vast set of networks of sensors and actuators? What kind of sociocultur- 
al theoretical approach would you suggest when encountering the technical and human move- 
ments and relations forthcoming and dwindling as interactivity in and around your national 
library, involving hundreds of thousands of human interactants on the move along with several 
million entities tagged with RFIDs? What kind of sociocultural theoretical approach would 
you adopt in the face of the multiplicities of technical and human movements and relations 
coming together and withdrawing interactively in and around JFK, LAX, or Frankfurt Airport, 
including those of human individuals and social groupings, mobiles, heating, ventilation, and 
lighting systems, transport systems, security systems, and airplanes? 
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As a researcher in hardware engineering, software development, or interaction design, 
would you be quite sure that your current studies of network topology, dynamic systems 
theory, along with certain recent sources treating of swarm intelligence, flocking, ant colo- 
nization, and a-life would be adequate to the kinds of complexity and emergence on the 
horizon for ubicomp cultures? The task at hand could be given a first delineation or set of 
contours by looking at some of the qualities of a ubicomp system listed by Chalmers et al: it 
will be fluid; its structure will vary in the short term and evolve in the long term; each non- 
human entity will be purposive, vaguely or formally; it will be partly autonomous - some of 
its interactions are determined by its purpose and its interactive experience rather than by 
invocation from a higher authority; it will be reflective - subsystems can report experiences 
to higher systems and perhaps humans to permit intervention or guidance (Chalmers et al.: 
2). From this approach it seems to follow that what takes place through interactivity in ubi- 
comp cultures will not just involve complexity but will do so in a strong sense, i.e., irreduc- 
ibly, and so interaction designs or not only to be engaged with as dynamic but as evolving 
(and decaying or dying out). 



This is closely related t the claims made by Matthew Fuller and Sonia Matos in their contri- 
bution to this issue. In 'Feral Computing: From Ubiquitous Calculation to Wild Interactions' 
the readers will find not only an explicit critique of the tendency to develop ubicomp as 
seamless and invisible by overemphasizing the machine and keeping the deployed human 
component stable and unexcited, thus replicating the military and Al strands of the compu- 
tational heritage from WWII onward so as to risk once again an obfuscation of the poten- 
tial for a different reworking of interrelations with the dynamics of thought, computational 
subjects, societies, as well as modes of life. The readers will also find a pursuit of a deeper 
understanding of human-computer symbiosis that will engage with a wider notion of sys- 
tems and ecologies. Notably, Matos and Fuller will seek to substantiate their claim that the 
complex and feral potential of ubicomp now (to be) actualized consists in a renewed and 
expanded context for computing, shifting from 'calculation' to 'interaction.' A environmen- 
tally distributed, situated, and embedded ubicomp is thus to be approached as an opening 
of new spaces for variable kinds of users, cognitions, and contextualizations that echoes a 
number of central aims of the second-order cybernetics worked out by Heinz von Foerster 
and others. This is not least a matter of an attempt to study complex systems, including hu- 
mans and machines, while attending to their formation of patterns of reflexivity, to their gen- 
eration of a recursive sense of self, and to the wider processes in and with which that self 
co-composed. Fuller and Matos thus also acknowledge ubicomp cultures and their interac- 
tion designs as a question of complexity - with a certain emphasis, perhaps, upon a strong 
version. For the authors' attempt to affirm a 'feral' ubicomp draws upon a notion of systems 
as analytically indecomposable (although they also mention systems qua a growth by par- 
taking of an expanded conversation, as well as different forms of structural couplings). Feral 
ubicomp would, then, be a matter of interactivity qua connectivist composition and decom- 
position involving dynamic relations among the entities and interactants in a wider and quite 
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multiplicitous ecology. Interaction design for ubicomp cultures would here correspond to think 
with things, or to start finding means of generating rigor in the chaotic, but doing so recur- 
sively, promising to produce something that renders the complexity of these ubicomp cultures 
sensible and open. 



This is by no means the only kind of approach, however, as one might surmise by looking at 
Shin'ichi Konomi and George Roussos' treatment of the ways in which ubicomp technics are 
finding their way into large scale real world commercial information systems (rather than 
small scale research simulations): 



The core enabler for ubiquitous computing is technology that makes the physical and 
digital worlds interlinked and thus intimately related. Every object in the world we live 
in has a digital representation that follows the situation of its real self and vice versa. 
This unique linking of bits and atoms opens up numerous possibilities for new comput- 
ing interactions which are currently explored by ubiquitous computing research. One 
of the main implications of this interlinking is that people, places and things acquire 
unique machine readable identities within systems of very large scale that must be ac- 
commodated within inflexible physical constraints and constantly changing usage con- 
text. To be sure, auto-identification capability opens up membership to ubiquitous com- 
puting systems for numerous entities and potentially results in massive increases to 
the number of constituent elements and system complexity. Understanding the issues 
raised by this increased complexity and exploring solutions can be hard to carry out in 
lab-based studies or case studies of limited scale. Such understanding often requires 
that experiments are carried out at scale, a fact that invariably implies high costs that 
are almost certainly prohibitive in a research context. Nor is it possible to identify and 
address such issues using large scale simulations, as these are limited by the simplified 
assumptions involved, and cannot take into account the emerging behaviors caused by 
real users. (Konomi and Roussos; 508) 

One observes that no matter which way one goes about it, one will have to draw upon notions 
of 'complexity' and 'emergence,' neither of which is so far readily available in a clearly defined 
sense. While Konomi and Roussos also identify and acknowledge the question concerning 
complexity and emergence, their project does not embrace strong nor irreducible complexity 
but remains explicitly devoted to a version of reductionism - something allegedly necessary to 
meet the economico-practical limits and needs of the real (commercial) world. In fact, reduc- 
tionism is both by far the most traditional response and a highly justifiable and respectable 
one, since otherwise most standards of scientific research and practical development are put 
in quite some doubt. 



Hence one can understand John Thackara's reaction in his book on designing in a complex 
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world (Thackara). In the name of a biomimicry of the lightness and seeming simplicity of 
natural phenomena and evolution, Thackara proposes that we reject and put aside the com- 
plexity of ubicomp cultures. He is quite plausible when criticizing some of the stronger claims 
made for ubicomp - that it will be seamlessly embedded in our environment, fully personal- 
ized, altogether self-adaptive, fully anticipatory of our needs, and deliver only information 
rich with contextually relevant meaning. But, however uncomfortable this is, I do not think 
the complexity of ubicomp can be reduced away easily, as Thackara implicitly calls for when 
remarking on the claims made for ubicomp: '...maybe I am missing something, but to me 
this translates as: Build systems that are too complicated to understand and then, after they 
are deployed, find ways to master their complexity. Hmmm' (204). He is no doubt right in his 
contemporary historical diagnosis: 'Trillions of smart tags, sensors, smart materials, con- 
nected appliances, wearable computing, and body implants are now being unleashed upon 
the world all at once. It is by no means clear to what question they are an answer - or who 
is going to look after them, and how' (198). He is also undoubtedly right that very few of us 
need to interact with an electronic toothbrush that harbors 3,000 lines of hard code. This 
does not entail, however, that one can avoid addressing interactions with the complexity of 
ubicomp culture as such, as complex. 



Maybe there is a sound ethical insight and point to be found in Thackara's treatment. In that 
case he shares that insight with Donald Norman who insists in his recent book on making a 
distinction between 'complexity' and something 'complicated' (Norman, 2011). 'Complexity' 
has to do with a state of the world, suggesting things with many intricate and interrelated 
parts, Norman argues. 'Complicated' has to do with a state of mind, including not least the 
secondary meaning of that which is 'confusing.' Maybe the two authors could be brought to 
agree, then, to Norman's suggestion that we forget about the complaints against complex- 
ity, only to complain against confusion, against whatever makes us feel helpless or power- 
less in the face of mysterious forces that take away control and understanding. On this view, 
bad interaction design for ubicomp cultures has no excuse, whereas good design can help 
tame the complexity, a complexity which is required but should be managed and be made 
understandable, according to Norman. This type of argument will not be helpful enough - 
assuming that it will not do to militate against complexity in the most traditional way, i.e., by 
insisting on granting priority to some version of reductionism (in this case Norman's insist- 
ence upon reduction to the understandable). This will be insufficient in case an affirmative 
pursuit of the complexity of ubicomp cultures is called for, and this will certainly be the case 
each time their emergent traits are to be addressed, no matter how uncomfortable it might 
be to admit of contingency and the spontaneously unpredictable. 



In fact, a more affirmative way to approximate a theoretical set of resources for dealing 
with complexity and emergence in this context can be found in a source as early as Warren 
Weaver's 1948 article on science and complexity (Weaver). This text might be said to present 
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several forward steps in that respect with its double distinction, first between 'problems of 
simplicity' (largely concerned with two variables) and 'problems of complexity'; then between 
problems of 'disorganized complexity' on the one hand (billions of variables as in physics and 
math, to be handled by probability theory and statistical mechanics), and, on the other, prob- 
lems of 'organized complexity' (the middle region concerning ways to deal simultaneously with 
a sizable number of factors which are interrelated into an organic whole). Perhaps Weaver's 
now classic article is particularly useful because it would allow us to see that almost all of our 
questions concerning the complexity of ubicomp culture and their interaction designs belong 
to such problems of 'organized complexity,' as are similar problems in biology, medicine, psy- 
chology, economics, and political science. Reduction to simply two variable problems is not 
possible, but it is also not a matter of applying a mathematics of averages to a huge number 
of factors. Not least, an approach via this middle kingdom of organized complexity might per- 
mit us to acknowledge the occurrence of emergence, i.e., that which Weaver already pointed 
to when saying that members of diverse groups can work together to form a unit which is 
much greater than the mere sum of its parts (542). 



Even though the contributors to this issue of the Fibreculture journal have different ap- 
proaches to this, the issue as a whole can be read as an insistent voicing of a concern with 
complexities of ubicomp cultures whose bundled relations of interactivity display emergent 
traits. The wager is that a great many kinds of interactivity in ubicomp cultures can be mod- 
eled or simulated as so many complex systems qua a non-linear coming together and disper- 
sal of diverse interactants and components. This necessarily means that interaction designs 
for ubicomp cultures are to quite some degree unpredictable and uncontrollable, not least for 
cybernetics as a science of control. But perhaps they are also positively emergent qua tenden- 
tially self-organizational (i.e., 'designing' means to afford local dynamics of interactions so as 
to eventually produce global coordination and synergy). Perhaps this remains at the heart of 
interaction designs for ubicomp cultures: they must be and remain a complexly moving tar- 
get. One will have to engage with the existing interaction designs and those to come so as to 
find out whether their interactive networkings are emergent in a strong and/or a weak sense. 
One will have to explore whether and how their spontaneous ordering of complex systems 
and patterns out of a multiplicity of relatively simple interactions do and do not go beyond 
the qualities of their components. It will be of some interest to see whether and how one 
might approach interaction designs for a ubicomp culture by thinking of and working with its 
interactive complexity in terms of scale-free networks or small world networks, as thought by 
Albert-Laszio Barabasi and others (Barabasi, 2002, 2009; Newman, Barabasi and Watts) as 
well as by Duncan Watts and Steven Strogatz (Watts and Strogatz). This is not least of inter- 
est because it already amounts to affirming a belief in a complexity of interaction design that 
cannot be strictly defined, only momentarily and locally placed as fluctuations somewhere 
between ordering of structures and dissipating into disorder (Nicolis and Prigogine; Prigogine 
and Stengers). 
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Notes 



[1] For one good, short, and relatively early account of the status quo of ubicomp research, 
including its limits and unresolved issues, see the article written by two computer scientists 
from Intel (Want and Pering). 



[2] Gregory Abowd and Elizabeth Mynatt provide one good, short historical overview from 
2000 of the implications for HCI of ubiquitous computing, such as natural interfaces, con- 
text-aware applications, and automated capture and access (Abowd and Mynatt). Jonathan 
Grudin's more recent introduction for the third edition of the Human-Computer Interaction 
Handbook (2011) offers a much broader and more detailed rehistorization of HCI whose 
closing part engages with ubiquitous computing under the rubric of discretion and invisible 
HCI (Grudin). 



[3] For a more extensive treatment of the notions of mixed reality at stake in current cultural 
theory along with research and development, see my introduction in Throughout (Ekman). 
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[4] A more detailed account of engagements with context-awareness can be found in my in- 
troduction to Throughout (Ekman). 



[5] See also David Tennenhouse's useful insistence on delimiting any traditional human-cen- 
tered vision focused on interactive computing now that we are approaching the networking of 
thousands of embedded processors per person on the planet (Tennenhouse). 
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Introduction 



Ubiquitous computing positions a world where computation is embedded into our surround- 
ing environment. Rather than retrieving information and communication from distinct devices 
(PCs) removed from contexts and activities, ubiquitous computing proposes that the mediated 
can become an integral part of our environment, seamlessly implanted, much in the way that 
text-based communications are incorporated now (Weiser, 1991). As such, ubiquitous computing 
places us in a world where material reality merges with computational reality, and where the 
built environment holds a strata for computation, a readiness towards the digital and the medi- 
ated. 



This paper asks what the architectural consequences of ubiquitous computing can be. Architec- 
ture is concerned with the designing and making of spatial contexts. The central objective of 
architectural design practice is to define spatial ambitions and ally these with the given design 
criteria. In this way it holds a different point of departure to ubiquitous computing. The aim for 
ubiquitous computing is to support the incorporation of interactive experiences in ways that 
can, for example, extend and support the existing framework of a work situation or the home. 
Where ubiquitous computing seeks models for embedding the digital in a meaningful and seam- 
less manner, it also privileges the idea of the everyday, of the environment and surroundings of 
its users as fundamentally antecedent to the digital tool. Reconciling the cultural and technolog- 
ical ambitions of ubiquitous computing with the practice of architecture therefore brings forth 
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new challenges for the development of concepts and techniques by which digital events can 
become "embedded", that is, can become part of the conception of architecture rather than 
an afterthought. 



This article reflects on the two interactive installations Breathing Room and Slow Furl, fol- 
lowing a practice-based research method. The paper explores the issues involved from two 
perspectives. Firstly, it queries how embedding ubiquitous computing and the idea of re- 
sponsiveness into architectural practice can lead to new models of interactivity. Referencing 
the concepts of self production and recursiveness in robotics and early cybernetics, it seeks 
not to fix the user in a situation of apparent control but rather to position the environment 
as a whole within a continual condition of self invigoration. We propose the term pulse as 
a measure of temporal extension, in order to think temporality and movement as part of an 
architectural language of design. 



The second perspective concerns the inherent challenge to the material practices of archi- 
tecture that result from working with responsiveness. Architecture has traditionally been cast 
within the image of the permanent and the static. Embedding the ability to sense and act is 
therefore one of several recent events in architecture that questions its tectonic conceptuali- 
sation. By proposing instability as a concept for material thinking this article references the 
emerging field of transitive materials. Materials are thought performatively both in respect to 
their structural and their computational facility. 



The idea of 'embeddedness' poses questions not only in respect to the use of and behaviour 
within a ubiquitous computing space or to the infrastructural necessities that this might re- 
quire, but also, more fundamentally, to what it means for computation to become an integral 
part of the built environment. The article asks the following questions. What happens to the 
relationship between the spatial conceptualisation and occupation of an environment when 
computation becomes an integral part of our built environment? How does a consideration 
of architecture as interactive challenge traditions of spatial conceptualisation and lead to a 
rethinking of core architectural concerns such as stability and structure? What happens to 
our material practices as the ability to sense and actuate become core properties of an 'ar- 
chitecture of change'? 
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Breathing Room and Slow Furl: Textile architectures oF actua- 
tion and response 



The ambition for the two architectural installations Slow Furl and Breathing Room is to im- 
agine a textile architecture that acts upon and reacts to changes in their environments. Con- 
ceived as robotic installations at an architectural scale the installations are speculative probes 
querying the territory of robotics, architecture and ubiquitous computing, while suggesting 
new spatial concepts for a soft space. 



The installations use textiles as a material for construction. A primary interest in textiles within 
these projects is their inherent pliability. As compressive materials, textiles allow for easy state 
shifts such as folding, pleating or stretching. But textiles are also interesting from a manufac- 
turing point of view. Textile technologies are also methods of material production: knitting. 




Figure 1: Slow Furl, textile view. 
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weaving, lacing, or crocheting are structural techniques by which single fibres are brought 
together to make fabrics, generating their particular properties of pliability and flex. Control- 
ling these techniques as well as the specific properties of the fibres allows the specification 
of the material performance. Understanding textiles as a technology as well as a material, 
the installations propose new ways of embedding the computational into the built environ- 
ment. 



In Breathing Room and Slow Furl the textile is understood as a matrix for embedding com- 
putation. By integrating conductive fibres into the textile membranes the architectural skin 
itself becomes part of the electronic circuit of the robotic environment. The installations use 
this externalised circuitry to create a strata of soft switches (Berzowska, 2004) by which the 
state of the fabric, its position and folds, can be registered. As the installations move, the 
conductive paths within the material change, thereby creating shifts within the electronic 
state. These state shifts act as a sensing matrix of material interaction. 



The membranes clad dynamic substructures, or armatures, that hold their particular motility, 
informed by the materiality and structure of their construction. The armatures are actuated 
using simple motors steered in turn by an array of micro controllers. The micro controllers 




Figure 2: Breathing Room, textile view 
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use the soft switches as input steering the actuation of their movement cycles. In Breath- 
ing Room and Slow Furl computation exists as a distributed system. In both installations the 
armatures are steered using multiple autonomous micro controllers connected to each their 
part of the strata of soft switches. The movement of the installations is therefore instigated 
as a set of independent actions and reactions in turn unified by the continuity of the textile 
membrane. In this way the building skin itself becomes part of the computation, effectuating 
the emergence of the complete movement score. 



The key constituent of the two installations lies with their programming. In both installa- 
tions interaction happens through material manipulation and state change. This material 
call and response is the basis for the programming of their responsive behaviours. The code 
is constructed as finite loops prompting the movement of the motors and armature. As the 
membranes fold and unfold they instigate changes within the circuitry thereby continually 
calling forth further movement cycles. Each movement loop is potentially the last. It is only 
through the material interaction, the actual self-touching of the membranes, that a continuity 
of movement is established. This continuity, the pulsing of switching and unswitching, creates 
the emergence of a behavioural patterning defining the architectural quality and presence. 




Figure 3: Slow Furl, textile view (2). 
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It is into this cyclical looping that the user enters. In Breathing Room and Slow Furl interac- 
tion is physical. Those engaging with the works are encouraged to manipulate the material 
through touch and movement. As they manipulate the physical arrangement of the mem- 
branes, shifting their positions and creating change in the externalised circuitry they affect 
and change the production of new movement cycles. The inhabitants and the environments 
become part of the same material interaction, both affecting and changing the movement 
scores of the structure. The presence of the user is conceived as an intensification, or, con- 
versely, a relaxing, of the continual movement of the installation. 

SelF production and the auto-poetic system 



The installations explore the idea of self-production. Rather than considering interaction 
as something the user instigates and controls, interactivity is understood as the means by 
which the installations iteratively self-produce their movement cycles and, through this, their 
behaviour. It is in this way that the installations propose new ways of understanding the 
relationship between user and environment, occupant and space. 



In architecture, the thinking of ubiquitous computing has become synonymous with build- 
ing automation projects such as the Intelligent Building or the Smart House. As such, the 
embedding of sensors enables the steering of the complex infrastructures of a building, such 
as heating, ventilation, and air-conditioning. It is now commonplace that buildings switch 
their heating or lighting on and off in response to its usage, and that more complex systems 
such as air-conditioning are steered by central computing units (Wiggington and Harris, 
2002; Compagno, 2002). Buildings therefore become interfaces that continually control the 
complex layering of exchanges between outside environmental conditions and the perceived 
needs of its user. To use the favoured analogy of the cyberneticians, the building becomes 
an advancement of the simple thermostat, a means by which the conditions of the living 
environment can be regulated and controlled (Weiner, 1965). 



While this model is useful for the control of complex building infrastructures, it also points to 
a particular understanding of the role of technology in architecture, a role both challenged 
and intensified through Breathing Room and Slow Furl. In conventional instances of ubiqui- 
tous computing, the subject, or the user model, becomes the centre of an interactive system, 
where every call demands a response. The environment then remains subservient to the 
user, creating a relationship of control and determinacy. 
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Yet in cybernetics, the causal model of the thermostat has been the cause of much debate. 
If Norbert Weiner understood the thermostat as a control unit generating the stability of a 
system, the second order cyberneticians developed a more complex model of the intercon- 
nection of system and environment, one in which a circular causality allows for inter-relation- 
al actions and reactions (von Glaserfeld, 2000). Here, the continual exchange of conditions 
between environment and system is understood as a mutual negotiation in which there is 
no primary cause or instigator. Instead of understanding the system as under a measure of 
control, environment and system are conceived as mutually dependent, continually affecting 
each other through their recursive interchanges. 



It is this circular relationship that Breathing Room and Slow Furl seek to engage. In Breathing 
Room and Slow Furl the aim is to create a position where user and environment are part of 
the same cycle of material interaction. The condition of self-touching (installation) and touch- 
ing (user) affects the same part of the circuitry thereby creating equivalence between action 
and interaction. 



While on one level the rethinking of causality challenges the primacy of the subject, it also 
proposes a more fundamental shift in understanding the central aims for the system. Where 
the causal model points beyond the system for the instigation of change, the recursive 
interactions of the interconnected system seeks stability as a result of the dynamics of the 
interchanges between system and environment. With this model of recursive interchange 
an idea of 'liveness' is introduced. The live system is one that can preserve its interactions 
and recursive interchanges between system and environment in ongoing self-production. As 
defined by the cybernetician and biologist Humberto Maturana, an auto-poetic system (both 
organic and mechanical) is one which continually reconstitutes itself through its iteration of 
its processes of production. Here, self-production becomes the key objective for the system 
for which everything that takes place in [the] autopoetic system is subordinate to the 
realisation of its autopoiesis' (Maturana, 1980: 149). Whereas the system is open to its envi- 
ronment, it is closed in its aim, or, as Ranulph Glanville puts it: while it is open to informa- 
tion, its organisation remains closed' (Glanville, 2004: 1381). 



Returning to an architectural perspective, this shift of focus fundamentally changes the origi- 
nal percept of the building as subservient to its user into a near converse construct of intro- 
spection and self absorption. In this optic, the building becomes concerned with itself, and its 
occupant changes from "user" to participant, intrinsically linked to the system and part of its 
cycles of self-production. 
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In the making of Breathing Room and Slow Furl the thinking of self-production as a means of 
interaction has lead to a similar state of introspection. In both installations the focus of in- 
teraction is to preserve the continual movement of its membranes. But this does not rupture 
the programmatic intentions of the installations. Despite existing as speculative probes both 
installations engage abstract programmes of occupation and use. 



Breathing Room was developed as a probe for an interactive scenography. The scenography 
was imagined for a dance performance and the folds and pleated gatherings of the fabric 
are scaled for the human body. As an extended backdrop, Breathing Room creates a shallow 
space that rhythmically expands and contracts opening up to its user, allowing the dancer to 
enter its complex skins. Slow Furl is an active wall membrane developed as a site specific in- 
stallation for the Lighthouse Gallery, UK. The membrane includes scaled deepenings that act 
as seats, inviting the user into its flowing skins. Both installations engage the user directly 
through a measure of programmed occupation, and the introspective and in essence autono- 
mous sense of interaction affects the design space of their conception. 



Clearly, these shifts imply profound challenges for the practice of architecture. If architecture 
is traditionally understood as containing its occupant, and architectural design practice is 
concerned with the conception of this frame, inscribing this process into a cybernetic un- 
derstanding of interaction and self-production changes architecture and its design from the 
making of an absolute specification to that of the devising of a dynamically changing sys- 
temic relationship. Thinking architecture as live, as motile and as self-sustaining, challenges 
both the object and the process of architectural making. 



Pulse as a Measure of SelF-Production 



In Breathing Room and Slow Furl the recursive interchange and self-production have lead to 
the conceptualisation of pulse as a way of understanding the temporal dimensions of the 
installations. The spatial intentions of the installations are defined across time, occupying 
time as an integral part of their performance. But this is not an abstract time of endless ex- 
tension. Rather it is a time of agency and action: an enacted time that is neither cyclical nor 
linear, but random and fractured, always potentially, and disastrously, final. 



The two installations are different in their way of engaging this temporal extension. Breath- 
ing Room was developed first. The installation is lightweight and transparent, its pulsing 
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tuned to the breath of its occupant. In Breathing Room the armature is made from steel rods 
controlled by a pulley system. Much as with a fishing rod, the pulleys cause the rods to bend 
in a spline shape. The armature is clad with a translucent organza fabric pattern cut to vary 
in density as it folds. The material performance of the fabric is further detailed through laser 
cut patterns that locally changes the translucency, weight and draping of the membrane. 
Into this building skin conductive fibre is embroidered, creating a complex patterning of soft 
switches. The switches are connected to the steel armature. As such the armature becomes 
part of the circuitry passing the state changes to the micro-controllers. The fabric and the 
armature are understood as parts of the same complex composite material - collapsing, 
sensing and actuation - in which the motor systems trigger the switching and unswitching of 
the externalised circuitry, causing a continual oscillation making the membrane contract and 
expand. 




Figure 4: Breathing Room, topological view 
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Figure 5: Breathing Room, topological view (2) 



In Breathing Room the space is highly motile. The lightness of the membrane, variegated by 
its detailing, gives the installation an ephemeral expression. The movement loops are like- 
wise agile, the pulse of the installation resonating with the pulse of its inhabitant. 



Slow Furl was developed after Breathing Room. As a reaction to the design of the temporal 
pulsing in Breathing Room, the aim for Slow Furl was to query the temporal dimensionality 
of a motile architecture. During the design of Breathing Room the tempo of each movement 
loop was discussed. Referencing former work (Ramsgard Thomsen 2007a, 2007b, 2008), it 
was noted that the timeframes of these installations had related to the size of the structure. 
Just like the quick breath of a small bird, the size of the organism relates to its breath or 
pulse. Critical to that reflection was the consideration of what would happen to this system 
at the scale of architecture, and how this could be related to the slow pulse of the inorganic. 
As such. Slow Furl explores a geological time of imperceptible flow. Like a glazier or a frozen 
river, the installation imagines a space of continual change taking place in times that exceed 
our immediate perception. 



As a structure. Slow Furl is in many ways similar to the Breathing Room installation. Slow 
Furl is defined by a thick spacer fabric cladding a dynamic armature which is moved by a 
series of stepper motors, that in turn are steered by a network of independent micro-control- 
lers. Armature and fabric are understood as part of a complex composite that merge sensa- 
tion and actuation. But where Breathing Room is ephemeral and agile. Slow Furl is thick-bod- 
ied and intransigent, moving as if in a deep pace at the edge of stasis. 



In Slow Furl the armature is made of a wood structure with a series of jointed arms pulled 
vertically by the motors. The installation is heavy-set; the wood armature weighing the struc- 
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Figure 6: Slow Furl, topological view 



ture down and the many joints generating friction and an inherent inflexibility. In contrast the 
motors are exceptionally weak. Running at 5 volts they move at a quick pace without much 
torque. In collaboration with an engineering company we developed bespoke gearboxes 
gearing down the pace of the motors and generating a slower but more forceful output. In 
this way Slow Furl uses little energy to create great strength at a deeply set slow pace. 



The movement of the installation is differentiated in areas of movement contrasted with 
areas of stasis. Its thick membranes meet the underlying armature, partly in a tight fit and 
partly through a voluptuous folding allowing for the movement of the armature. In Slow Furl 
the triggering of movement happens through the same structure of movement loops as in 
the Breathing Room installation. The membrane is embroidered with conductive fibres creat- 
ing an externalised circuitry that as the installation moves triggers new movement cycles. 
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The occupant enters this self-interaction. Invited to touch, or to sit within the deep folds of 
its pleated membranes the user reconfigures the externalised circuitry. However, because 
of the slow speed of the movement, users have little means of understanding their effect 
on its slight pace changes. Users therefore remain almost entirely outside an experience of 
interaction, relating more to the structure's material presence than its responsiveness. 




Figure 7: Slow Furl, textile view (3) 



Challenging Material Practices: Looking For the instable 



As practice based probes the two installations query the material-tectonic realisation of a 
motile architecture as well as its conceptualisation. The idea of embedding computationally 
steered events into building skins presents profound challenges to the material practices 
of architecture. As architecture becomes a material for ubiquitous computing it becomes 
necessary to develop a new material foundation that allows for sensing and actuation. 
This presents a key shift in architectural practice away from the stable, towards the mo- 
tile suggesting new structural concepts by which the state-shifting can be integrated as 
an inherent part of material performance. Breathing Room and Slow Furl thus investigate 
the making of 'transitive materials' that combine the structural performance of traditional 
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material approaches with new computationally-informed properties. Transitive materials are 
inherently transformative. They are: 'material organisations containing computation(s) and/ 
or promoting unique digital fabrication methods as elements inherent to their properties and 
behaviour' (Coelho et al, 2007: 2) The interest in structural and material consequences of an 
architectural configuration of ubiquitous computing is informing a growing number of archi- 
tects. In the work of practitioners such as Philip Beesley (Beesley, 2007), Omar Kahn (Kahn, 
2009) and Neri Oxman (Oxman, 2008), spatial design takes its point of departure in material 
design investigating the potentials for actuation and movement, whether at the scale of the 
joint or in the actual compositing of materials themselves. 



In Breathing Room and Slow Furl the membranes merge sensing and actuation into one 
complex skin. Here, textiles are understood as a material approach, a means by which the 
designed interactivity can be materialised. Conceived as composite materials, the mem- 
branes collect skin and substructure, merging the inherent pliability of textiles with the motile 
performance of the armatures. Rather than building for permanence, the installations seek to 
engage the material performances of the organza and steel, spacer fabric and wood, so as 
to create a platform for movement and change. The installations are therefore in very funda- 
mental ways informed by the material presence of their structures. The motility and behav- 
iour of the installations are shaped by the pliability of their cladding fabrics, the flex of the 
steel rods (Breathing Room) and the weight of the wood structures (Slow Furl) as much as by 
their programming and motor movement cycles. 

Conclusions 



Breathing Room and Slow Furl explore how ubiquitous computing and the idea of embedded- 
ness pose profound challenges to architectural thinking. Rather than seeing the embedding 
of computation as an augmentation or extension of existing models of space and inhabi- 
tation, these projects ask what happens as computation becomes an integral part of our 
environment. The installation's twofold focus on interaction design as well as material design 
presents a complex weave of exchanges between the computational and the material. In 
Breathing Room and Slow Furl interaction is uniquely tied to the embodied presence of the 
material. The interactive system not only relies on outside stimulation, but also on its own 
material presence for continual invigoration. The externalised circuitry is a network by which 
the continual process of self-production takes place. The interactive system is always poten- 
tially ending, only re-finding its pulse as new connections in its membranes are found. 



Breathing Room and Slow Furl also inquire into the material reality of a ubiquitous comput- 
ing space. Using textiles as a technology as well as a material, the installations investigate 
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how the design of a material performance can becomes part of spatial design. Rather than 
understanding the material as an existing strata for the embedding of computation, the in- 
stallations ask how particular techniques lend themselves towards material specification that 
can be locally defined both in respect to its structural and its computational properties. 



A further consequence of this thinking is the extension of architecture into the temporal. The 
installations ask how this temporality can be architecturally stratified. Time is understood 
through a model of agency, continually enacted and lived. Where Breathing Room seeks to 
imitate the breath of its user. Slow Furl questions the temporality of interactive experiences. 
Where computers normalise the instantaneous feedback between the call of the user and 
the response of the system. Slow Furl asks what happens when this relationship is scaled up. 
By engaging a geological time. Slow Furl introduces concepts such as resistance and intran- 
sigence as ways of understanding the relationship between the moving and the formed. 
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Introduction 



This article treats the philosophical underpinnings of the notions of ubiquity and pervasive 
computing from a historical perspective. The current focus on these notions reflects the ever 
increasing impact of new media and the underlying complexity of computed function in the 
broad sense of ICT that have spread vertiginiously since Mark Weiser coined the term 'perva- 
sive', e.g., digitalised sensoring, monitoring, effectuation, intelligence, and display. Whereas 
Weiser's original perspective may seem fulfilled since computing is everywhere, in his and Seely 
Brown's (1997) terms, 'invisible', on the horizon, 'calm', it also points to a much more impor- 
tant and slightly different perspective: that of creative action upon novel forms of artifice. Most 
importantly for this article, ubiquity and pervasive computing is seen to point to the continuous 
existence throughout the computational heritage since the mid-20th century of a paradoxical 
distinction/complicity between the technical organisation of computed function and the hu- 
man Being, in the sense of creative action upon such function. This paradoxical distinction/ 
complicity promotes a chiastic (Merleau-Ponty) relationship of extension of one into the other. 
It also indicates a generative creation that itself points to important issues of ontology with 
methodological implications for the design of computing. In this article these implications will 
be conceptualised as prosopopoietic modeling on the basis of Bernward Joerges introduction 
of the classical rhetoric term of 'prosopopoeia' into the debate on large technological systems. 
First, the paper introduces the paradoxical distinction/complicity by debating Gilbert Simon- 
don's notion of a 'margin of indeterminacy' vis-a-vis computing. Second, it debates the idea of 
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prosopopoietic modeling, pointing to a principal role of the paradoxical distinction/complicity 
within the computational heritage in three cases: 



a. Prosopopoietic aspects of John von Neumann's First Draft of a Report on the EDVAC from 
1945. 

b. Herbert Simon's notion of simulation in The Science of the Artificial from the 1970s. 

c. Jean-Pierre Dupuy's idea of 'verum et factum convertuntur' from the 1990s. Third it con- 
cludes with a brief re-situating of Weiser's notion of pervasive computing on the basis of this 
background. 



First part - Pervasive Computing - Beyond Use as Interaction 



One of the best kept secrets of new media success in the past twenty years or more is the 
creative role of the human user. This claim may outrage any serious observer of new media - 
after all, isn't new media about human user interface, user interaction, media (user) culture, 
appealing graphics and applications (to the user) and so on? What about Web 2.0 and social 
software? Isn't the history of computer use since the 1940s one long process of accommoda- 
tion of the human user in still more sophisticated manners? 



The short answer is no. 



The creative action of human use as a principal instance of human ontology, that is an entity 
that may be different in principle from computed function, yet important for such function, 
has never been queried in the history of computing. The conception of computed function 
has never been approached from the perspective of a form of Being ontologically different 
from computing, yet of crucial importance. Often it has simply been ignored. Think about the 
ongoing thrust of etymology: the term for socially inclusive software, 'social software' - net- 
worked communities of humans - is still 'software', and so on. The predominant assumption 
(and a tenet of a basically cybernetic origin) has been that human use is ontologically predi- 
cated on implications of the technical organisation of computed function. In fact, in the his- 
tory of computing the human has grosso modo been rendered an entity to be ontologically 
surpassed or even eradicated by the diffusion of computed function into all aspects of the 
human's Being. From computer chips in model trains, intelligent buildings, and smart materi- 
als, to robotics, to models of the brain's functionality, all sorts of computer based modeling, 
and further, to recent ideas of a connectionist network sociology (Watts, 2003; Barabasi, 
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2003); the attempt to fully implicate the ontology of the human Being within computing has 
been significant. 



A preliminary sketch may indicate three common frameworks for understanding human use 
in this regard (use/interaction etc.) in the post war era: 



(a) Human use derived as directly as possible from computed function in well 
defined measures of design (e.g., Jakob Nielsen's 'usability' [Nielsen, 1993]), 

(b) Use derived from perceptual aspects of approaches to computed function 
(e.g., Donald Norman's 'affordance states' [Norman, 2001 [1998]]), 

(c) Use derived from computed function as social context (e.g., Susan Leigh 
Star's assumption of new media as 'community building media' [Starr, 1995]). 

Put differently, the technical organisation of computed function has been posited as onto- 
logical determination of use in the sense of human use by a human Being, that is, of this 
Being's body, psyche-soma, intelligence, behavior, emotions, social predication and so on. In 
sum, this is use in whatever form we know. 



Pervasive Computing - Creating an 'Un[<nown ArtiHcia! World' 



The broad origin of this determinism is what we, for want of a better term, may call a com- 
putational heritage originating in the mid-20* century (from Alan Turing's response to David 
Hubert's 'Entscheidungsproblem' in the 1930s onwards). This heritage today encompasses 
a stunning mass of issues, disciplines and knowledge closely related to formalisations (e.g. 
algorithms), languages/codes (e.g. programming languages), techniques (e.g. CPUs), ar- 
tifacts (e.g. computers) and smart materials (e.g. piezo-materials). In addition to this are 
further perspectives arising within new disciplines and theories such as self-organisation, 
connectionism and complexity - all contributing to a paradigm of 'complexification' (Casti, 
1994, 2002; Eve et al., 1997; Mainzer 1994; Watts 2003; Barabasi, 2003). Added to this, there 
are the immense and diverse movements into the 'posthuman' (Hayles, 1999) along with 
constructivist social and cultural theory and philosophy, exemplified by the thought of Niklas 
Luhmann and Gilles Deleuze, to mention two well-known thinkers. Further, there is the ap- 
pearance of 'classic' (Toronto School) and new media and communication theory (Michelsen, 
2005, 2007). 
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However, considering the scope of computed function in today's society there is an impend- 
ing need for a look at the human Being in different terms. Put differently: if the claim of 
pervasive and ubiquitous forms of embedded digital computing is correct what does that 
entail for the human Being which 'embeds' this? To begin to answer this, I turn to a different 
understanding of the artificial. 



Ezio Manzini has argued for an entirely novel state of the artificial per se. He argues that 
there exists a discrepancy between the human creation of the artificial and the lack of 
insight into the distinctive character of the resulting effects. 'To man the artificial is a com- 
pletely natural activity,' (Manzini, 1994: 44), but the resulting artificiality appears as 'an un- 
known artificial world that we must examine to discover its qualities and laws' (Manzini: 52). 
Manzini argues that it is necessary to establish positive connotations for the artificial. 



In this article the current interest in pervasiveness and ubiquity will be used as occasion for 
discussing such connotations, beyond an ontological determination by computed function. 
The article's point of departure is Cornelius Castoriadis's idea of a new form of self organisa- 
tion pertaining to the Human Being per se: 'a human strata of the real'. This strata is distin- 
guished by a strong ontological imagination - 'the imaginary institution of society' (Castori- 
adis, 1987). All phenomena within such a strata reside ontologically within human creativity, 
what he terms 'humanity's self-creation': 



Humanity self-creates itself as society and history - there is, in humanity's self-crea- 
tion, creation of the form of society, society being irreducible to any 'elements' what- 
soever ... This creation takes place 'once and for all' - the human animal socializes 
itself - and also in an ongoing way: there is an indefinite plurality of human societies, 
each with its institutions and its significations, therefore each also with its proper 
world. (1997: 339) 

In Domaines de I'homme (1986), Castoriadis writes that a new type of distinction/complic- 
ity between ratio and creativity follows from the idea of humanity's self-creation. He argues 
that it is necessary to undertake the project of distinguishing between and thinking together 
the 'ensidic [rational] dimension and the proper dimension of the imaginary' (1986: 16-17). 
Folllowing from this, it is possible to envisage an ontological 'distinction/complicity' between 
the technical organisation of computed function enacted in a human strata of the real and 
a strong ontological imagination of a human Being (Michelsen, 2007). There is a need to 
undertake a project of distinguishing between and thinking 'together' the ratio of computed 
function and the proper dimension of the imaginary in an ontological sense of creative ac- 
tion upon such function. 
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For all these reasons, the following will account for what I have called a paradoxical distinc- 
tion/complicity between the technical organisation of computed function and the human Be- 
ing in the sense of creative action upon such function. This paradoxical distinction/complicity 
will be described as the opening of a certain form of aisthesis based on the intertwining of 
function and creative action as two instances which are ontologically distinct, yet deeply 
complicit. This is what I will term a chiastic aisthesis, following Merleau-Ponty's famous 
idea of the 'chiasm' of the 'visible and the invisible' (Merleau-Ponty, 1983). Furthermore, the 
approach here will be conceptualised in general terms as a methodology of prosopopoietic 
modeling developed from the classical rhetorical device of prosopopoeia 
(Greek: npoocononoua). 

Pervasive Computing - Paradoxical distinction/complicity 



A first observation of the paradoxical distinction/complicity may be found in the work of 
the French pioneer of cybernetics, Gilbert Simondon, from 1958. Simondon writes about the 
relation between the aesthetic and the cybernetic organisation of 'technicity,' what he terms 
the 'ensemble' (Simondon, 1989 [1958]) (all translations of Simondon's French text are mine. 
A. Michelsen). For Simondon, the problem of 'the aesthetic' has implications for certain acts 
of primary importance. The aesthetic form of thought, the aesthetic object - fundamentally 
the aesthetic , as he argues in a dense overview - points to a peculiar origin, a 'remarkable 
moment' [point remarquable] or 'position', 'event', of cultural issues (Simondon, 1989 [1958]: 
179ff). The aesthetic suggests a 'bifurcation' of the natural world by way of a remarkable 
moment which makes explicit a cultural world of humans, and further a human form of Be- 
ing. Importantly, this world stands out as a particular bounding of the aesthetic in a human 
mode which is non-dichotomous, i.e. not within the received idea of a subject-object dichoto- 
my. It is a form of organisation relying on a dimension of Being - the cultural, the human etc. 
- which is in-between 'pure objectivity and pure subjectivity' (187). 



The idea of the ensemble thus indicates a different organisation bounded not by computed 
function but by an appraisal of the human which is open towards computed function. This 
allows one to conceive of the human in an ontological sense which is on one hand different 
from the technical organisation of computed function, and on the other hand related to this 
organisation. It is possible to envision two ontologies without creating a conflict with the 
cybernetic tenet, because they refer to principally different forms of organisation which are 
nevertheless open to each other in the sense of an intertwining in-between 'pure objectivity 
and pure subjectivity'. 
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For Simondon it is important to avoid a definition of the ensemble (11) as an extended de- 
termination by computed function. On the contrary, a computing machine exhibiting a range 
of options is not an automat determined by a vocabulary of closed functions. It is a machine 
which qva its technological definition introduces a principal margin of indeterminacy relying 
on human intervention, defined by Simondon as 'perpetual innovation:' (12) 



(...) the being of the human intervenes as regulation of the margin of indeterminacy, 
which in the final consequence is adapted to a more comprehensive exchange of 
information. 

The technical organisation of computed function thus resides within an enforcing openness 
because of innovative human steps emerging in the margin of indeterminacy. We may argue 
this way: at one extreme of a spectrum the technical organisation of computed function is 
'hard' and almost impenetrable for the human, e.g. in the aggregate automated workings 
of code. At another extreme the human is almost wholly aloof in a self-enclosed imaginary 
flux which has nothing to do with forms of computing. However, in the intertwining middle 
there emerges an enforcing openness between computed function and creative action. Here 
computed function becomes defined by the margin of indeterminacy and creative action ren- 
dered computational. 



Computed function is thus co-defined by a 'regulation' in the shape of 'perpetual innovation', 
that is, integrally related to the different instance of a human Being. This is what is unac- 
knowledged or figured as ontologically surpassed in the computational heritage. 



Simondon thus indicates a paradox. There exists a paradoxical distinction/complicity be- 
tween the technical organisation of computed function and the human Being in the sense of 
creative action upon such function. In one perspective they are wholly distinctive; in another 
they are complicit. The result is that computing as we know it reside within a paradox. 



Simondon thus argues not only for a paradoxical distinction/complicity beyond ontological 
determination of the human Being by computed function. He further indicates that: 



a. The distinction/complicity entails a margin of indeterminacy in between the 
purely objective and the purely subjective, i.e., beyond modern Western dualism. 

b. The ensuing 'regulation' can be seen as an intertwining of two distinct forms 
of organisations at the emergent level of a bifurcation. 



52 FCJ-131 fibreculturejournal.org 



Anders Michelsen 



c. Here, cybernetically speaking, 'a more comprehensive exchange of informa- 
tion' is possible between innovative - or creative, forms of action, and the tech- 
nological organisation of computed function. 

d. All that has to do with use is appearing in this intertwining middle. I want to 
suggest that it does so as a specific form of aisthesis. 



From Pervasive Computing to CIniasm 



This can be taken further by involving the notion of the 'chiasm,' outlined in Maurice Mer- 
leau-Ponty's famous draft of what was to be the posthumuous work of The Invisible and the 
Invisible (Maurice Merleau-Ponty, 1983 [1968]) from the same decade as Simondon's work 
on the technical object 



Here Merleau-Ponty elaborates the famous idea of the visual as a 'flesh' of the world based 
on a 'chiasm' of something 'visible' and 'invisible'. While his well known early work on the 
'phenomenology of perception' from the 1940s argues that 'our own body is in the world as 
the heart in the organism: it keeps the visible spectacle constantly alive, it breathes life into 
it and sustains it inwardly, and with it forms a system' (Merleau-Ponty, 1986: 203), he takes 
radical steps in the second posthumous phase which allow for thinking the aisthesis of a 
paradoxical distinction/complicity: 



With the first vision, the first contact, the first pleasure, there is initiation, that is, 
not the positing of a content, but the opening of a dimension that can never again 
be closed, the establishment of a level in terms of which every other experience will 
henceforth be situated. The idea is this level, this dimension. It is therefore not a de 
facto invisible, like an object hidden behind another, and not an absolute invisible, 
which would have nothing to do with the visible. Rather it is the invisible of this world, 
that which inhabits this world, sustains it, and renders it visible, its own and interior 
possibility, the Being of this being. (Merleau-Ponty, 1983; 151) 

As research into the 'paradox' in the 20* century has shown - notably Willard Van Orman 
Quine's investigations (Quine, 1966), a paradox does not rule out meaning per se. Some 
paradoxes, Quine argues, are only insolvable in their time. That is, they are only to a certain 
extent anomalies, e.g., Zenon's paradox. At a later stage they may be verified, and thus be- 
come veridical, even if some still maintain - so far - their anomaly. The chiasm in our context 
may be reviewed as a paradox under its way to verification. 
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This is exactly what Merleau-Ponty argues. The chiasm is a verified paradox based on the 
transversal structure of the flesh which organises the visible in forms of meaning beyond 
the seeing subject vis-d-vis the visible object. Moreover, this is a form which in its intertwin- 
ing becomes inner and outer in the same moment, that is, a fact of the world and an idea. 
It is an organisation, or an ensemble, which organises potential and actualises forms by an 
inherent dynamic involved with the intertwining. 



We can look at the history of computing from this perspective. Take the history of the inter- 
net. From the first drafts of internet architectures in the late 1950s and 1960s by Baran and 
Davies, up to the breakthrough of the World Wide Web in the mid-1990s, important initia- 
tions of creative action upon computed function have emerged. In Janet Abbate's words, 
the network architectures developed from the 1950s onwards did not only accommodate a 
variety of computed functions, they were also paradoxically open in a very productive sense 
Their later success was closely connected to this feature, 'the ability to adapt to an unpre- 
dictable environment' (Abbate, 1999): 



No one could predict the specific changes that would revolutionize the computing 
and communication industries at the end of the twentieth century. A network archi- 
tecture designed to accommodate a variety of computing technologies, combined 
with an informal and inclusive management style, gave the Internet system the abil- 
ity to adapt to an unpredictable environment. (Abbate; 6) 

From the Home Brew Computer Club (1975-1977) and its focus on amateur designs for a 
centralised technology that moved away from corporate, military and research uses, to the 
spread of cell phone technology in the past decade, such unpredictable environments have 
played a massive role for any new formation of use. Every turn of use has presupposed and 
implied perpetual innovation in the sense indicated by Simondon, i.e., human use - what 
Abbate, perhaps somewhat vaguely, terms 'an unpredictable environment.' And vice versa, 
every instance of human use has triggered a huge development of computed function. In 
direct contrast to many assumptions of the computational heritage, the human has not been 
an ontological entity to become eradicated. It has bounced back with every cusp of new 
use, to the point where current new media growth is almost wholly reliant on a deep human 
presence, as we see in phenomena such as Facebook and Youtube. 



It is possible, indeed necessary, to argue that every time computing has leaped forward with 
new successes, this has resided paradoxically within an instance of the Human Being in the 
sense of creative action upon computed function in a still more comprehensive manner. 
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The current focus on the notion of pervasive computing is no exception. Notions such as 
'pervasive', 'ubiquituous' and 'embedded' are yet another example of Simondon's para- 
dox, emergent as chiasm. This is clearly apparent in Mark Weiser and John Seely Brown's 
manifesto The Coming Age of Calm Technology from 1997 (Weiser and Seely Brown, 1997). 
Here they envision a 'third wave' of computing - 'beyond calculation' in the Turing machine 
sense - which will be more at the users disposal. It will take on a membrane-like, active set 
of relationships between artifact and human, which, at the time of writing (1997), is primar- 
ily 'post the personal computer of the desk top (PC)': 



The Third wave of computing is that of ubiquitous computing, whose crossover 
point with personal computing will be around 2005-2020. The 'DC era will have lots 
of computers sharing each of us. Some of these computers will be the hundreds 
we may access in the course of a few minutes of Internet browsing. Others will he 
embedded in walls, chairs, clothing, light switches, cars - in everything. (Weiser and 
Seely Brown: 77) 

In the ""UC" era' there will emerge a chiastic relationship between almost omnipresent 
computed function -in everything, experienced as seamless by the user. 'Lots of computers' 
will be 'sharing each of us'. There will emerge a membraneous interaction between new 
ubiquituous artifice - an 'artificial environment' (Manzini) - on the basis of which two forms 
of organisation will chiastically engage with each other. This will be what Weiser and Seely 
Brown discuss as the absent, yet omnipresent 'calm' which 'engages both the center and 
the periphery of our attention and in fact moves back and forth between the two' (Weiser 
and Seely Brown: 79): 



... it should be clear that what we mean by the periphery is anything but on the 
fringe or unimportant. What is in the periphery at one moment may in the next mo- 
ment come to be at the center of our attention and so be crucial. 

This is 'encalming' (80) because: a. It allows us 'to attune to many more things' since 
computed function is at a distance yet present, that is, absent yet present, 'the periphery 
is informing without overburdening', and b. It allows us to operate in-between center and 
periphery, thus placing the peripheral ressources of computed function in a 'calm through 
increased awareness and power:' 



... without centering, the periphery might be a source of frantic following of fashion; with 
centering, the periphery is a fundamental enabler of calm through increased awareness 
and power, (italics my emphasis) 
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Despite Weiser and Seely Brown's explicit reference to the debates within HCI, e.g., Gibson, 
Norman etc, around issues such as affordance, what is suggested here is in fact in almost 
complete accordance with Simondon's idea of a particular bounding of the aesthetic in 
between 'pure objectivity and pure subjectivity' (Simondon, 1989; 187). One may even ar- 
gue, without taking the parallel too far, that the actual event of encalming is exactly the 
emergence termed by Simondon as 'perpetual innovation'. Moreover, this entire conjecture 
is deeply chiastic, almost to such an extent that it may be translated directly into Merleau- 
Ponty's draft description of a transversal structure of the flesh which organises the visible in 
forms of meaning, beyond the seeing subject vis-d-vis the visible object. This flesh is sim- 
ply the dimension which allows for the intertwining middle described by Weiser and Seely- 
Brown. The options enabled by DC make for a chiastic organisation residing with this dimen- 
sion. 



Second Part - A Methodology oF Prosopopoietic iviodeling 



In the second part of the paper I will debate this chiastic organisation in the sense of its 
methodological implications for the design of computing, inspired by Bernvard Joerges's 
interesting note on the need for a prosopopoiesis of large technological systems (e.g., like 
the internet) (Joerges, 1996). With prosopopoiesis Joerges refers to the rendering meaning- 
ful of large technological systems in a human mode. Prosopopoiesis will be seen as a proper 
methodological definition of what I have termed a paradoxical distinction/complicity be- 
tween the technical organisation of computed function and the human Being in the sense of 
creative action upon such function. It follows from the verified paradox of the chiasm, based 
on a transversal structure of flesh. It opens up certain approaches to novel forms of arti- 
fice. Importantly, this establishes positive connotations for the artificial in the terminology 
of Manzini. That is, it encourages the type of designs that allow for the chiasm that Weiser 
and Seely Brown discuss. In blunt terms, it allows for the designs which will render computed 
function a billion user large success in the second half of the 20"^ century. 



Put differently, there exists a real chiasm - intertwining, of the paradoxical distinction/com- 
plicity - which makes up a history of use, i.e., creative action upon technological organisa- 
tions of computed function (in the last resort, of the universal Turing machine idiom with all 
that it entails). This is guided methodologically by prosopopoietic modeling. 
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Prosopopoietic Modeling - Methodology 



Joerges argues that large technological systems institute a 'Macht der Sache' [power of 
things] in a social context which needs to be rendered meaningful in their technical organi- 
sation by a 'showing' (15ff) in a human mode as 'Korper der Gesellschaft' [embodiment of 
society]. For this he uses the term I have already begun to discuss, 'prosopopoiesis,' derived 
from the notion of prosopopoeia in classical rhetorics. A prosopopoeia is a rhetorical device 
in which a speaker or writer communicates to the audience by speaking as another person or 
object. Joerges gives an example of this in the technological organisation by which the rain 
embodies the roof in the appearance of a delicate sound pattern intelligible to the aisthesis 
of a human underneath, who becomes enactively aware of the technical organisation of the 
roof (265ff). The rain appears via a particular gestalt - a prosopopoeia - that makes sense to 
a human: a sound to ear and so on. 



Large technological systems must enact a response in the user in order for the systems to 
function (had the roof-user not been able to identify the roof, he might not creatively have 
found shelter, so to speak). This is paradoxical in the sense that it transforms the technical 
organisation into a much different but no less relevant range of emergent events involv- 
ing the system 'roof and sound. The fall of rain on the roof is thus potentially full of mean- 
ing which the prosopopoiea expands poietically by creative action. This literal implication 
is needed in order to situate the system fully. The roof has to become situated creatively as 
meaning for humans in order to function. Or more radically, the function of the roof is imme- 
diately situated within a plenitude of creative meaning and there is not really any function, in 
fact no roof, without it. 



Prosopopoiesis may thus be described as a method for modeling events within Simondon's 
paradoxical distinction/complicity as a margin of indeterminacy opening for innovation of a 
range of possible uses. To be precise, it situates use as any form of human use. Or, differ- 
ently, it situates the negotiations between Weiser and Seely Brown's center and periphery in 
something which may be rendered a method for enabling specific designs. Prosopopoiesis 
becomes a methodological locale for creative action upon computed function where we may 
discover the details of the chiasm intertwining 'pure objectivity and pure subjectivity' (Simon- 
don). In the following I will pursue some further indications in three steps: 



Prosopopoietic modeling - John von Neumann's Draft: the basic traits of prosopopoiesis, 
discussed via Paul Ricoeur's idea of heuristic inscription and the complexity of calculus in the 
Turing machine. 
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Prosopopoietic modeling - Herbert Simon's simulation: a response to prosopopoiesis in the 
computational heritage. Here I will focus on Simon's idea of simulatory interface. 



Prosopopoietic modeling - verum et factum convertuntur: negotiations from within the 
computational heritage: Here I will discuss Jean-Pierre Dupuy's observation of modelling as 
product and transcendence of human finitude. 



Prosopopoietic iviodeling - von Neumann's Draft 



According to Wikipedia, 'A prosopopoeia (Greek: npoaajnonoiia) is a rhetorical device in which 
a speaker or writer communicates to the audience by speaking as another person or object' 
(Wikipedia. The Free Encyclopedia: Prosopopoeia): 



This term also refers to a figure of speech in which an animal or inanimate object is 
ascribed human characteristics or is spoken of in anthropomorphic language. Quintil- 
ian writes of the power of this figure of speech to 'bring down the Gods from heaven, 
evoke the dead, and give voices to cities and states ... 

In order to enact meaning vis-d-vis an 'inanimate object,' the object in question must bring 
about a gestalt which may relate to the creative action of a human. In the computational 
heritage, this basic trait may be said to have first emerged as a creative action upon the 
complexity of calculus in the Turing machine. The Turing machine was one of the most so- 
phisticated artifacts ever invented, not least because of its potential for modelling everything 
compatible with the universal Turing machine form. Yet it also appeared difficult for humans 
to approach directly. A number of complicated steps had to be taken in order to understand 
and utilise the machine, from the construction of storage techniques to the development of 
programming as such (Ceruzzi, 1998). The user, early on and later, had to establish a number 
of procedures going beyond anything hitherto seen. 



Concrete computed function (e.g., the chaotic cablings of the ENIAC, the first general-pur- 
pose electronic, digital and reprogrammable computer from the mid-1940s) had to be crea- 
tively related to use by a heuristic force of fiction, residing within a figure of speach which 
might introduce, 'a general function of what is possible in practice' (Ricoeur, 1994: 127). Use, 
in other words, was not a piece of cake. Creative action upon the machine facing the user 
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for a long time required something close to a complete reinvention of it. This was the first 
encounter of prosopopoietic modeling. It was not a haven or a promise, but a downright 
requirement. And if computing today may appear more 'plug n' play' it is only because of 
decades of toiling and juggling, not only with specific forms of prosopopoeia, but with an 
excruciating construction of whole fields of interrelated prosopopoeia - what to the user ap- 
pears seamless as 'the web.' 



The first appearance of prosopopoiesis was consonant with the first programming of the 
ENIAC and other early Moloks of pioneering computer design by an engineered intuition of 
how to e.g. combine cables in order to calculate. Despite, or exactly because of the eccentric 
expertise required for these operations, they immediately posited the issue of prosopopoietic 
modeling: of methods to enact chiastic aisthesis. 



A famous early instance is John von Neumann's draft of a principal logical design of a com- 
puter. He wrote this as the First Draft of a Report on the EDVAC, from 1945, after a few days 
of inspection of the of the ENIAC. In order to grasp the portent of this machine von Neumann 
applied a 'figure of speech,' by describing the ENIAC's technical organisation of computed 
function via an instrumentarium of logic. Von Neumann saw through the technical mass of 
relays and combinatorics, to a momentous and automated logic figuration - i.e., a future 
logic that was the precondition for making explicit the universality of the Turing machine. 



This was a radical rethink of the ENIAC-inventors' (Mauchley and Eckert) accomplishment, 
in terms of what is still known as the 'architecture' of the serial computer. The mechanical 
workings inside the 'black box' of Mauchley and Eckert's ENIAC - the 'inanimate object,' i.e., 
the concrete mechanically instantiated calculus of the Turing machine, were by von Neu- 
mann's logic tables opened towards a methodological coupling of Mauchley and Eckert's 
mechanical engineering with logic in the sense of creative action upon these mechanics. To 
paraphrase Paul Ricoeur, mechanical workings were opened to a mode 'in which we radi- 
cally rethink what family, consumption, government, religion and so on are' (Ricoeur: 132). 
Logic was thus applied as creative action, not only upon the specificity of the ENIAC in 1945 
but upon the universal Turing machine, via a chiastic aesthesis which offered the special- 
ised wartime artifact of the ENIAC the broader promise indicated by Turing. Prosopopoiesis 
may thus be taken as another term for a creative heuristic of the universal Turing machine, 
attempting to tackle methodologically the paradox of distinction/complicity that any user 
would encounter. 



From the very beginning of the computational heritage - for example as summarised in the 
emphasis on universality of the universal Turing machine, it was clear that the mechanical 
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calculus was implying a new ratiocination. In this, human proof in the traditional sense of 
compelling human embedded logic was superceded by embedded technology. When it came 
to the Turing machinics - mathematics and algorithmics embedded in digtal relay-structures, 
the machine could in plain words calculate much more than any human and, importantly, in 
modes with a comprehension, scope and intrinsic purport which a human could only, at best, 
address intuitively. 



That was the birth of mechanical approaches to 'complexity' which until then had been a 
vague metaphor (Merriam-Webster dates complexity to 1661, at the time of the origin of the 
modern ratio in a sense somehow of mechanical computing, c.f. 'reckoning' in Hobbes's us- 
age from the same time). It came into being, we may argue, because the instrumentarium of 
logic was methodologically applied as prosopopoietic modeling. To put it bluntly, the com- 
puter posited to the world from the beginning a paradoxical dinstinction/complicity between 
a mechanically embedded machinics much beyond any human grasp which was neverthe- 
less deeply reliant on human creativity. 



In fact the paradox was so compelling that the rendering operative of the Turing machine 
was only possible when creating it heuristically by way of prosopopoiesis. The Turing ma- 
chine - as artifice, would only make sense if prosopopoietically modeled, whether in Turing's 
own explanatory metaphor of the famous 'tape', or by Mauchley and Eckert's concrete 
cablings, or by von Neumann's logic tables in the First Draft of a Report on the EDVAC, or- 
consequently, in any other way created by humans. 



This machine could only be entered into a general function of what is possible in practice for 
humans once such a prosopopoietical method was established. There was no way around it. 
Had this not happened, we may polemically ad, the whole issue of mechanical computation 
could have remained an eccentric side effect of the American wartime economy. Indeed oil 
the more or less eccentric historical precedents had turned out to be exactly this - Pascal, 
Babbage, Lovelace etc., up to the advent of Turing, Shannon, and von Neumann. 



The success of computing would raise the issue of from where and how to create: of how to 
procede, grasp, deal, to imagine creatively, the artifact of computed function 'first' invented 
by Mauchley and Eckert's mechanics. And further, this prosopopoiesis would remain chal- 
lenging for the coming decades, in abject contrast to any naive idea of technique as 'ready 
to hand' (Heidegger would revise his assumptions in the postwar essay on technology). 
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In the first years of computing this task was left in the concrete to the cybernetic pioneers - 
often in quite practical ways, when fumbling with chaotic cablings such as the ENIAC's in the 
mid-40s, and later in the 1950s and 1960s extending this to a new expert culture of cybernet- 
ics and systems theory. 



By then the prosopopoiesis relating to the creative actions of experts had for a long time been 
involved in wild speculations which would imagine in diverse and increasingly public ways the 
Turing machine calculus as a 'general function of what is possible in practice.' Animated ma- 
chines, electronic brains, an entire world system trespassing the realm of God in the making; 
all of this as a 'mode in which we radically rethink what family, consumption, government, 
religion and so on are.' Not for nothing is the title of Norbert Wiener's last book God & Golem, 
Inc. A Comment on Certain Points Where Cybernetics Impinges on Religion (Wiener, 1964). 



Thus prosopopoietic modeling appears as a very first methodological precondition for making 
sense of an inanimate object, or better, for an apparant 'anima' of its Turing machine calculus 
(!). However, such deliberate fancy would enact the 'planned rationalism' of a technical or- 
ganisation of computed function into a 'situation' of much broader human use in the coming 
decades, in the words of Lucy Suchman writing when user-centric design was clear in sight 
(Suchman, 1987) 



A first description of the prosopopoiesis needed to methodologically situate the Turing ma- 
chine calculus is thus also a first heuristic force of inscription by creative action upon comput- 
ed function, no more, no less - from the outset a chiastic aisthesis, as indicated by Simondon. 
The very first inventions of specific ways of computing (from Konrad Zuse's electromechanical 
machine Z3 and the other 'firsts' to the ENIAC) had from the very outset to be re-invented 
as 'a general function of what is possible in practice' (Ricoeur, Op.cit.: 127) by paradoxically 
rethinking what is taking place inside the complexity of Turing machine calculus by way of 
prosopopoietic modeling. The cybernetic pioneers struggled bravely to get everything right. 
Yet they also testified to the dynamic of prosopopoiesis in their creative production of proso- 
popoeia, of methods relying on the creative enaction of figures 'of speech in which an animal 
or inanimate object is ascribed human characteristics or is spoken of in anthropomorphic 
language' (Wikipedia). And over the coming decades all sorts of paradoxical issues which had 
little to do with computing senso strictu came into being, as seen in examples from Kubrick's 
2001: A Space Odyssey (1968) to William Gibon's Neuromancer (1984) (Bukatman, 1993). See 
the graphic below for a summary of the argument: 
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Prosopopoietic Modeling - Simon's Simulation 



At this point we may argue that prosopopoietical modeling has several features: 



a. It enables a chiastic aesthesis, emerging in-between two distinct yet intertwined 
ontological forms of organisation, the Simondonian pure subject and object (see 
the part to the right in the graphics above). 

b. It moves the emphasis of use away from the technical organisation of comput- 
ed function in whatever complex form, and toward a heuristic force of inscription 
which makes relations possible by methods derived through prosopopoietic mod- 
eling. 

c. It makes possible a new approach to understanding use in this regard, in the 
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computational heritage, by moving towards creative action upon computed func- 
tion and a re-evaluation of the ontological form of the human Being. 



A further step of my argument may be presented by taking up Herbert Simon's discussion of 
'simulation at the front end of use' in the programmatic treatise The Sciences of the Artificial 
(Simon, 1996 [1969]). Here, it may be argued, heuristic inscription is tackled via the an- 
nouncement of methods of simulation. Simon argues that the computer makes up a particu- 
lar form of complexity which cannot and may not be put wholly at the disposal of the user. 
What is needed is a 'meeting point - an "interface" in today's terms - between an "inner" 
environment, the substance and organization of the artifact itself, and an "outer" environ- 
ment, the surroundings in which it operates' (Simon 1996 [1969]: 6). 



To Simon the intrinsic complexity of the inner environment of the artifact (i.e., the resources 
of the universal Turing machine) need not be wholly disclosed. Appropriate measures of 
simulation at the front end directed towards the user's attention may be enough. This argu- 
ment is in strict accordance with the deterministic positions of the computational heritage: 
the technical organisation of computed function is simply too comprehensive for any form 
of prosopopoiesis beyond simple simulatory presentations. That is, beyond what would be 
known from the 1960s as GUI, or 'graphic user interface,' in combination with other interac- 
tive devices such as mouse, keyboard etc, based on Ivan Sutherland, Douglas Englebart's 
and others experiments throughout the 1960s. In the late 80s and 90s this would evolve into 
the comprehensive design issues of usability, affordance and context, further user-centricity 
etc. 



Simon's solution thus appeared to fix prosopopoietic modeling through strict methods for 
displays of the technical orgnisation of computed function. At the time of his writing, in 1969, 
such methods were well under way to become commonplace, exemplified by new norms of 
e.g. usability. What would be less clear to Simon and the computational heritage was that 
this simulation was a very restricted form of chiastic aisthesis prosopopoietically subsumed 
under displays of the technical organisation of computed function. It would result in a bland 
and ironic substitute for the complexity of the Turing machine calculus, parading over dec- 
ades as the desktop. It was of course still a creative action upon the technical organisation 
of computed function, but it left the intrinsic complexity at the mercy of a prime metaphor 
from 19th century bureacracy, the desktop - or, in different words, linear print as a frame for 
enacting complexity. 



While affirming the basic traits of prosopopoietic modeling, simulation nevertheless became 
a veil to cover the paradoxical distinction/complicity, merely bearing witness to the fact 
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that the technical organisation of computed function would far supersede the human ca- 
pacity for ratiocination, leaving it at the front end as the infamous 'dumb' end user. But it 
was prosopopoietic modeling nevertheless. Simon's book can be read as a methodological 
manifesto which affirms the prescient assumption of chiastic aisthesis by Simondon, if by 
disregard. 



Prosopopoietic jviodeling - Verum et Factum Convertuntur 



The reduction of prosopopoietical modeling to a graphic simulation of a desktop would be 
quite convincing to prime users of computed function throughout the 80s and 90s. Nev- 
ertheless, even in this it demonstrates the paradoxical distinction/complicity between the 
technical organisation of computed function and the human Being in the sense of creative 
action upon such function in fully fledged operation. Today, however, this modeling appears 
as an increasingly daft entry to fast internet access etc. Simulation will hardly fit the context 
of radical heuristic inscription and the surging variety of prosopopoeia which have emerged 
throughout the 1990s and 2000s. 



By 2011 the cell phone, wireless and devices such as iPods, smart phones etc. have long 
made widespread prosopopoietical modeling possible, as we see in such phenomena such 
as web 2.0 and social software. It becomes apparent that the technical organisation of com- 
puted function today is working only when allowing for extended creative action. 



In his book on the origins of cognitive science Jean-Pierre Dupuy considers indirectly the 
importance of the new perspectives of modeling which appeared with the universal Turing 
machine calculus. Dupuy places it in a more reflected relation to the issue of prosopopoietic 
modeling. (Dupuy, 2000) 



Dupuy emphasises the importance of complex 'modeling' discovered by the computational 
heritage with a profound reference to Giambattista Vico's dictum of 'Verum et factum con- 
vertuntur.' This means that humans can 'have rational knowledge only about that of which 
we are the cause, about what we have ourselves produced' (27-28): 



A model is an abstract form ... that is embodied or instantiated by phenomena. Very 
different domains of phenomenal reality ... can be represented by identical models. 
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which establish an equivalence relation among them. A model is the corresponding 
equivalence class. It therefore enjoys a transcendent position, not unlike that of a 
platonic Idea of which reality is only a pale imitation. But the scientific model is man- 
made. It is at this juncture that the hierarchical relation between the imitator and the 
imitated comes to be inverted. Although the scientific model is a human imitation 
of nature, the scientist is inclined to regard it as a 'model,' in the ordinary sense, of 
nature. Thus nature is taken to imitate the very model by which man tries to imitate it. 
(29-30) 

The creation of a complex model is paradoxically both a product and a transcendence of hu- 
man finitude. The model abstracts from phenomenal reality 'the system of functional rela- 
tions,' putting aside everything else, and in its mechanical sphere of calculus, by computed 
function, such a model comes to obtain a life of its own, ' (...) an autonomous dynamic inde- 
pendent of phenomenal reality' (31). 



Now, according to Dupuy, this principle of Verum et factum gains a particular emphasis from 
the 1930s onwards. With Alan Turing and Alonso Church's alignment of computation and me- 
chanics a new significance of the artificial is conjectured, the issue of 'effective computability:' 



It seems plain to us now that the notion of effective computability that was being 
sought [in the 30sJ, involving only blind, 'automatic' execution procedures, was most 
clearly illustrated by the functioning of a machine. It is due to Turing that this mechani- 
cal metaphor was taken seriously. In his remarkable study of the development of the 
theory of automata, Jean Mosconi makes an interesting conjecture about the nature 
of the resistance that this idea met with in the 1930s; 'Considering the calculating 
machines that existed at the time - the perspectives opened up by Babbage having 
been forgotten in the meantime -, any reference to the computational possibilities of a 
machine was apt to be regarded as arbitrarily narrowing the idea of computability ... If 
for us the natural meaning of "mechanical computability" is "computability by a ma- 
chine," it seems likely that until Turing came along "mechanical" was used in a rather 
metaphorical sense and meant nothing more than "servile" (indeed the term "me- 
chanical" is still used in this sense today to describe the execution of an algorithm).' 
(35) 

Thus the ideas of Turing and Church not only expand on the notion of computed function, 
they also further the notion of what sort of technical organisation computed function can be 
seen to be. This is the road which takes the universal Turing machine into forms of modelled 
complexity, and further to a paradigm of 'complexification' (Casti) - highlighting the paradoxi- 
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cal distinction/complicity. Verum et factum embodies an acceptance of an integral need for 
prosopopoeia. It also seems to indicate that complexity modeling is only an underlying as- 
pect of prosopopoietical modeling, of a generalised, methodological, and in fact little reflect- 
ed creation of figures, 'of speech in which an animal or inanimate object is ascribed human 
characteristics or is spoken of in anthropomorphic language.' 



The machine may create something, not in its capacity for incorporated mathematics or in 
its capacity of calculating mechanics (Turing machine calculus in various forms), but in its 
capacity of effective computation, thus foregrounding 'effect' in terms of models which also 
reside chiastically with the human. Not only the machine but also the human will henceforth 
have all the options of computation as 'an autonomous dynamic independent of phenomenal 
reality' - i.e., as an element of a chiastic aisthesis. For all its capacity for effective compu- 
tation, any technical organisation of computed function is only meaningful in the methods 
which will effectuate its Turing universality as 'Verum et factum': ontologically transcending 
and ontological relying on creative action upon computed function. 



In Conclusion - Weiser's Notion oF Pervasive Computing Today 



Reading through my notes above, I think Mark Weiser's pursuit of pervasive computing takes 
on a specific quality. Texts such as 'The portable Common Runtime Approach to Interoper- 
ability' (1989), 'The Computer for the 21st Century' (1991), 'Libraries are more than informa- 
tion: Situational Aspects of Electronic Libraries' (1993a), 'Some Computer Science Issues in 
Ubiquitous Computing' (1993b), 'The Last link: The Technologist's responsibilities and social 
change' (1995), and 'Designing Calm Technology' (1995) read like an engineer struggling to 
expand the technical organisation of computed function into something else. Indeed Weiser 
himself points to the challenge 'to create a new kind of relationship of people to computers, 
one in which the computer would have to take the lead in becoming vastly better at getting 
out of the way so people could just go about their lives' (Weiser, 1993b: 1). 



For one thing, this captures the fact that they expose the paradoxical distinction/complic- 
ity between the technical organisation of computed function and the human Being in the 
sense of creative action upon such function. The machines must be designed so that crea- 
tive action is rendered possible. Or differently: there is no design, no machine without crea- 
tive action. This is a huge step away from Simon's simulatory strategies, once in accordance 
with Dupuy's argument for 'verum et factum', as discussed above. In order to transcend the 
finitude of the technical organisation of computed function, the technical organisation must 
pervasively transcend the realm of human finitude as ever-present, yet absent - made opera- 
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tive by prosopopoietical modeling. 



Second, however, this move away from the determinism of the computational heritage is only 
a partial step. It also installs a benevolent super-determinism, which in the period after the 
mid-90s appears increasingly outdated. The constant development of prosopopoietical mod- 
eling restates the paradox in its own way. While the engineering aspects are focused on quite 
restricted locales - the artificial definition of lap tops and cell phones etc., we see billions of 
prosopopoiea which simply appear as a blunt cultural instituting of new media. 



What we see in all this, I believe, is a 'cultural turn' of new media, one which by now is gen- 
erally acknowledged, and which has turned the engineering of Weiser, not to mention the 
simulatory techniques of GUI into quite useable but minor issues of computed function today. 
In this respect Weiser appears as a highly sympathetic engineer addressing us from a finished 
history: or better a history which from the beginning was misconceived, since all technologi- 
cal development resides with the paradoxical distinction/complicity. The world now indicated 
is a social world of new media, the perspetive of which we have just begun to see. But it is 
a world that increasingly exploits the paradoxical distinction/complicity that for so long was 
a problem for the computational heritage, and which continues to highlight prosopopoietical 
modelling. 



I will close by advocating an increased interest in such, e.g., in regard to new generations of 
new media for the developing world, where a cultural paradigm may make wonders of use 
(Michelsen, forthcoming). That is, there is a much keener focus on the options appearing 
when starting out from creative action upon computed function, not, vice versa, in whatever 
form and capacity. If the Turing machine in the mid-20th century was still a real enigma to be 
translated by heavy expertise, real enigmas today may reside much more with the latter part 
of the distinction/complicity: that is, with our enigmatic and human creativity. 
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Introduction 



As I write this, at the end of 2010, it is sobering to reflect on the fact that over a couple of 
decades of explosive development in new media art (or 'digital multimedia' as it used to be 
called), in screen based as well as 'embodied' and gesture based interaction, the aesthetics of 
interaction doesn't seem to have advanced much. At the same time, interaction schemes and 
dynamics which were once only known in obscure corners of the world of media art research/ 
creation have found their way into commodities from 3D TV and game platforms (Wii, Kinect) 
to sophisticated phones (iPhone, Android). While increasingly sophisticated theoretical analyses 
(from Manovich, 2002 to Chun, 2008 to Hansen, 2006, more recently Stern, 2011 and others) 
have brought diverse perspectives to bear, I am troubled by the fact that we appear to have ad- 
vanced little in our ability to qualitatively discuss the characteristics of aesthetically rich interac- 
tion and interactivity and the complexities of designing interaction as artistic practice; in ways 
which can function as a guide to production as well as theoretical discourse. This essay is an 
attempt at such a conversation. 



Over the first (roughly) two decades of practice, interactive aesthetic strategies were developed 
and adapted to the constraints of digital technologies — themselves under rapid development 
through this period — and substantial technical R + D as well as aesthetic research, was under- 
taken by artists. In fact, I propose that the reality that technical and aesthetic developments 



72 FCJ-132 fibreculturejournal.org 



Simon Penny 



were inseparable was a defining characteristic of the work of that period. Less obviously, 
digital and interactive practices were (only) slowly assimilated into the corpus of fine-arts 
practices and cross-fertilised with more traditional aesthetic approaches. 



Within the context of arts practices, a space for development of interaction less over- 
whelmed by the instrumental individualistic modalities of interactivity characteristic of the 
computer industry opened up, informed by generative systems and artificial life and dis- 
courses of emergent complexity and self-organisation which link back to cybernetic concep- 
tions of responsive systems. These negotiations were characterised by a not entirely clearly 
elucidated swing between preoccupation with user subject experience and a (more sculp- 
tural?) emphasis on the behaviour of the artefact, which offers less instrumental and more 
bio-mimetic approaches. [1] 



The domain of Interactive Art functioned over this period as a space of free invention, an an- 
archic research realm less application-driven and relatively free of market directives. In this 
context, in many cases, ideas appear long before they were recognised as research agen- 
das addressed in academic and corporate contexts. In that period, all kinds of experimental 
interaction modalities were realised by artists — many deploying custom technologies in code 
and/or hardware. The necessity to develop tools was double edged — on the one hand it per- 
mitted an organic development process in which the specifications of the technologies arose 
from theoretico-aesthetic requirements (and sometimes vice versa). On the other hand, the 
implicit task of running an engineering R+D lab with limited technical skills and usually very 
limited budgets was fatiguing. Without recognising these special conditions, it is not possible 
to grasp the significance of work arising in that period. In some cases, such work remained 
unknown outside a relatively closed tech-art community, and was independently recreated in 
other contexts. In other cases, the transfer was more explicit and sometimes it resembled a 
plundering. We might usefully examine the period to discover the roots of ubiquitous technol- 
ogies and the motivations for such. Over the last decade, much of the substance of interac- 
tive art research has found its way into commercial digital commodities and thereby suffered 
a confusing erasure of its historico-aesthetic significance. It is an odd feeling to see, in a few 
short years, systems which were perceive as having rich aesthetico-theoretical presence trivi- 
alised as mass produced commodities deployed in a paradigm of vacuous 'entertainment'. 



Echoing the fundamental computer science hardware/software binary of computer science, 
mainstream digital discourses were undergirded by a commitment, stated or unstated, to 
a dualistic polarisation of materiality and the 'digital', especially in the early years. This led 
to a deep and polarising discursive tension with the embodied and holistic perspectives of 
traditional fields of practice, rendering one camp 'luddite' and the other 'techno-fetishist'. 
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Recognition of the centrality of the negotiation of materiality and embodiment within digital 
practices is, in my opinion, fundamental to understanding the history of interactive art, and 
provides a purchase with which to understand transitions to ubiquity. The advent of cheap, 
'over-the-counter' technological bundles (microcontrollers, sensors, programming environ- 
ments...) has made the practice far more amenable to technological novices, and has also 
created a new set of aesthetico-technological challenges in the sense that these commod- 
ity widgets are designed to fit a narrow consumer need and thus have all kinds of decisions 
built into them (image 'improving' algorithms, digital video formats...) that are often difficult 
to isolate, let alone work-around. One of the happy upshots of the transitions outlined in the 
points above is the increasing presence of work by a second generation of artists in the field 
which more fluidly combine, for instance, material aesthetics of (even formalist) sculpture 
and installation with digital and interactive systems. Some examples of such work are given 
later in this paper. 



Interactivity and tine Emergence oF Digital Cultural Practices 



I identify a two-decade period — roughly speaking 1985-2005 — as the pioneering experimen- 
tal period of (computer based) interactive art. Crucial to the understanding of work in this 
period is the blindingly rapid development of the technological context. At the beginning of 
the period the graphical user interface was a novelty, the internet barely existed, the web 
was a decade away, interactivity was an intriguing concept. The production of acceptably 
high resolution illusionistic digital pictures (still frames) was an active research area and a 
megabyte of RAM was something luxurious. 



The period neatly brackets the emergence of most of the major technological milestones 
which now undergird digital culture and ubiquitous computing: WYSIWYG, digital multime- 
dia, hypermedia, virtual reality, the internet, the world wide web, digital video, real-time 
graphics, digital 3D, mobile telephony, GPS, Bluetooth and other mobile and wireless com- 
munication systems. It was a period of rapid technological change, euphoria and hype. 



In what follows I discuss several works of the period which in my mind, stand as markers for 
significant moments in the development of interactive digital cultural practice. I have chosen 
these works in part because I have experienced them directly and in most cases, I am privy 
to both the goals of the artists and the internal workings of the systems. I include three of 
my own works here, because in developing them I was engaged in the development of ideas 
and approaches in question, and also because I happen to know intimately the motivations 
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and development of designs and prototypes in these works. [2] 



There are numerous other works one might usefully discuss. It is a regrettable fact that, due 
in part to rapid changes in the technologies and the absence of appropriately skilled and re- 
sourced staff in appropriately set-up collections, I estimate that 75% of this work built in the 
last 25 years is lost entirely, and only a very small percentage of it persists in working form. 
It is sobering to reflect that a photograph 100 years old, if kept dry, is perfectly accessible 
to view, but the vast majority of commercial digital systems only a decade old are unusable, 
to say nothing of the many custom systems. [3] Occasionally, a heroic effort is conducted to 
rescue such work before it decays completely. The exhibition Eigenwelt der Apparatwelt by 
Woody Vasluka for Ars Electronica in 1992 was one such case in which works were rescued 
from basements and attics and restored. [4] 



Conspicuously, I here avoid discussion of desktop-based works. My reason for this is princi- 
pled. It has always been my opinion that such work — whatever its creative and experimental 
value — on the level of interaction, interface, and physical instantiation, took too much for 
granted. That is, by willingly adopting the constraints of the commodity desktop interface, 
assumptions about the nature of interaction which were reified in physical and system archi- 
tectures were implicitly endorsed and solutions appropriate for sedentary mathematico-sym- 
bolic deskwork were uncritically adopted. This, in my opinion, has had the effect of perturb- 
ing and constraining creative possibilities in regrettable ways. 



Interactive Art before the PC 



While the notion of a performative and processual aesthetics of interaction has been ban- 
died about for twenty years or so, looking at interactive art over that period reveals little in 
the way of development in the formal qualities of interaction per se. The interactional logic 
captured in Edward Ihnatowicz' Senster of the early 70s remains paradigmatic. [5] Senster 
also neatly framed the agendas of reactive robotics, biomimetic robotics, social robotics, 
twenty five to thirty years ahead of the institutional curve. Around the time of the Senster, 
Myron Krueger pioneered machine vision-based embodied screenal interaction in several 
works, the most well-known being Videoplace (1975). There is little in the Wii or the Kinect 
which was not prototyped in the several iterations of Videoplace forty years before. [6] 
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Figure 1. Senster on display at the Philips Evoluon, 
Eindhoven, 1970-1974 



In terms of autonomous behavioural repertoire, Grey Walter's Turtles of the late 1940s set a 
standard for machine behaviour seldom exceeded since. These devices, built with minimal 
(and by today's standards, rudimentary) technology, displayed behaviours seen today in 
artworks and robotic toys. The turtles were made not as artworks but as cybernetic experi- 
ments into electronic 'brains'. It is important to recognise that, consistent with the cybernetic 
context he worked in, the conception of 'intelligence' which Walter sought to simulate was 
thoroughly situated and pre-cognitivist. [7] The complexity of the creative agency of Gordon 
Pask's Musicolor of the late 1940s is likewise seldom attained (Pask, 1971). The Turtles and 
Musicolor were entirely analog, and Senster had the digital processing power, roughly, of an 
Arduino. 



These works then, provide a pre-digital or at least pre-consumer-digital reference point for 
interactive art. It is important to recognise that they arose within the discursive context of 
cybernetics — as opposed to the cognitivist regime of late C20th computing — and prior to the 
overwhelming presence of the digital commodity industry. And if this perspective explains 
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to some extent their inventiveness, then we might have to view the last quarter century as a 
period of increasingly successful attempts at rehabilitation from the disembodying discourses 
of the cognitivist and the 'digital'. 



It has been observed that the advances in Artificial Intelligence over the period of roughly 
1965-1985 can be attributed almost entirely to advances in hardware engineering — faster pro- 
cessors and more RAM have permitted larger scale brute force search, etc., as over the period 
formal procedures and basic techniques did not advance significantly. This assertion seems to 
have some justification, given that the computer upon which Deep Blue's chess program ran 
in 1997 was ten million times faster than the Ferranti Mkl upon which Dietrich Prinz's chess 
program ran in 1951. [8] 



Could it be that the design and aesthetics of interaction are subject to the same conditions of 
increase of technical capability without significant development of ideas? The rapid advance 
in bells and whistles permits interfacial cosmetic niceties undreamable 20 years ago. [9] The 
techno-fetishism of higher resolutions and faster bit-rates serves the needs of an industry 
which depends on obsolescence (as perceived or as inbuilt material breakdown) to remain 
profitable, often deploying novelty to obscure a void of significant advancement. The 'all the 
time, everywhere' catchphrase of pervasive computing (like the prospect of the same trivial 
and pathetic 'content' consuming more bandwidth as HD and 3D television) can be seen as a 
marketing device for bit-peddlers. Increasingly, physical devices are loss-leader market cap- 
ture devices, as printers are for ink cartridges. 



Grounding Interaction 



Having elucidated some of the historical realities of the practice, in this section of the paper 
I will outline some of the theoretical challenges. The first is also historical and has to do with 
sites of practice. Much of the work arose in the context of the plastic and visual arts, and as 
such, a theoretical void was encountered (or, more regrettably, not encountered). That is to 
say: whatever the theoretical tools available to address matters of form, colour, expression, 
and embodied sensorial engagement, those traditions had little to say about ongoing dynamic 
temporal engagement because traditional art objects do not behave. Within the traditions of 
aesthetics of the plastic arts, these are fundamentally novel issues. Recognition of this reality 
suggests an as-yet incomplete project of identifying prior fields of practice which have some 
theoretical relevance to this realm. [10] Questions like 'how does the act of interaction mean?' 
and 'how are such valences to be manipulated for enriched affective practice?' are fundamen- 
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tal questions in the aesthetics of interactive art. 



Long ago R. Buckminster Fuller (1970) asserted 'I seem to be a verb' and this cybernetically 
inspired locution was itself informed by the doubly continuous nature of the analog electronic 
signal: as temporally continuous and amenable to infinite resolution. The sentiment expressed 
by Fuller also informed much of the emerging art practices of that period which emphasised 
process — performance, site specific art of various kinds, etc. More recently the terms dynami- 
cal, processual, procedural, performative, situated, enactive and relational, each in their own 
milieu, capture the evanescent process of contextualised doing. There appears then to be a 
groundswell in the theorisation of cultural practices sympathetic with phenomenological ap- 
proaches, and by extension with emerging post-cognitivist cognitive science. I see this trend 
here as a critical paradigm shift which has major implications for the theory and practice of 
interactive art. 



Being a radically interdisciplinary realm, scientific and humanistic narratives collide in interac- 
tive art and the science wars have been a constant backdrop to work and theorisation in the 
field. Yet to the extent that interaction is an embodied process, neat separations of biology 
and culture are collapsed. My position here is evolutionary and materialist: interaction makes 
sense to the extent that it is consistent with or analogous to the learned effects of action in 
the 'real world'. Our ability to predict, and find predictable, behaviours of digital systems, is 
rooted in evolutionary adaptation to embodied experience in the world. We are first and fore- 
most, embodied beings whose sensori-motor acuities have formed around interactions with 
humans, other living and non-living entities, materiality and gravity. We understand digital 
environments on the basis of extrapolations upon such bodily experience-based prediction. 
This is easy to understand in mimetic environments such as Second Life, but is equally present 
in more basic mouse-screen levels of interaction. 



Our most common interactive modalities subscribe to or enlist assignments or connections 
which are deeply sensorimotoric in nature, and perhaps draw upon DNA hardcodings. Aside 
from the trivially Pavlovian modality (press the button and get the reward/food pellet) what 
are the key interactive modalities in artworks? In installation and robotic work, many exam- 
ples exploit a zoomorphic 'puppy paradigm' of approach and withdrawal, trust and fear. This 
is always beguiling — for a while. Is the charm of this modality somehow 'natural' to us as 
humans, perhaps hardcoded into our DNA as parenting animals? This question opens a field 
of inquiry at the intersection of neuroethology and interactive aesthetics. Whatever the case, 
the next question is how to move aesthetic development of the field beyond this biological or 
cultural 'ground zero'. 
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It is a seldom noted corollary to the panegyric around 'the virtual', that many interactive art 
projects focus on the dynamics of embodied interaction (Penny, 2011). This central aspect of 
human being-in-the-world was conspicuously poorly addressed in conventional sit-at-a-desk 
computer systems. Because of the sensitivity of artists to persuasive sensorial immediacy 
and embodied engagement, interactive art practice pioneered research into dimensions of 
interaction which remained opaque to institutional and commercial labs for many years. In 
this spirit, one must acknowledge that people making interactive art were doing 'affective 
computing' a decade ahead of academic and industrial recognition of such issues. 



In the contemporary context, this situation has changed in two ways. Interface technologies 
are far more diverse, complex and subtle. Not long ago, microphones and cameras were 
exotic add-ons to computers. The embedded miniaturised accelerometer has become ubiq- 
uitous and has contributed to the development of all sorts of gestural and body-dynamic 
driven applications. [11] And yet, first generation interactive modalities involving pointers 
and keyboards hang on skeuemorphically. Of all the things I do in my life, only some of them 
map well onto sitting at a desk in front of a glowing surface, poking at buttons, nor is this 
situation improved one iota when the context is miniaturised so the buttons are smaller than 
my fingers and I have to put my reading glasses on to look at the screen. For all the expan- 
sion of wireless networks (etc.) we have not progressed very far in interactive modalities. As 
in Al, advancement in the field can, as often as not, be attributed to advances in hardware. 



More subtly, the first generation to have lived with digital devices during infancy is now ap- 
proaching adulthood. This generation acculturated to, for instance, multi-modal on-screen 
interfaces. These people's neurology must have, to some extent, formed and developed 
around such systems. That is, the metaphors and behaviours of digital systems, like any as- 
pect of language and culture inculcated in infancy, have generated isomorphic neurological 
structures — digital metaphors instantiated at the level of cellular biology. [12] 



Who or What Is Interacting? : Analysis oF Interactive Systems 



Conversations regarding the aesthetics of interaction sometimes take on a weirdly schizo- 
phrenic quality due to the fact that some speak from the point of view of user experience 
and some from the point of view of system design. The question 'is it interactive?' can have 
wildly different answers depending on this point of view. One can maintain, as some do, that 
viewing a photograph is 'interactive'. Such positions are clearly nonsensical if one is look- 
ing from the perspective of the artefact/system, and are destructive of the goal of building a 
richer critical discourse about interactive systems. The photograph does not change in any 
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way due to changes in its environment. A human viewer might have varying experiences due 
to personal associations, varying proximity or lighting conditions, but there is no interaction 
in the sense of an ongoing sequence of mutually determining actions between two systems 
possessing agency, or as interacting components (user and machine) in the larger user/ma- 
chine system. The bifurcation in such conversations is whether the critique addresses the 
experience of the 'user' or the behaviour of the system. 



While there is undoubted value in probing the nature of the interactive aesthetic experience 
on the part of the subject, study of the design of the system as an armature upon which the 
experience occurs must provide a necessary complement. Such a design-centric approach 
engages issues such as designer authoriality and the position of the system as a literature — 
as discussed by Mateas and as is becoming central in some aspects of software studies 
(Fuller, 2008). Other (Artificial Life) approaches address the system as quasi-organism, in 
autopoietic or enactive sensori-motor loops with user(s) (etc.) (Penny, 2010). 



To focus on the machine, the fundamental requirement of an interactive system is that it 
correlates in a meaningful way, data gathered about its environment (usually a user's be- 
haviour) with output. That is, the system must present effects which are perceived by the 
user as being related to their actions. Without this there is no perception of interactivity. 
From the perspective of system design, successful interaction comprises two mirrored parts: 
first, the sensing and interpretation aspects of the system must gather relevant information 
about the world and interpret it 'correctly'. Secondly, action of the system must be contrived 
such that they are perceived to be related to the events in the world which were sensed and 
interpreted. Meaningful interaction thus requires that several functions be correct and coordi- 
nated correctly. Sensors must be chosen correctly and calibrated correctly to capture relevant 
environmental electro-physical variables and such data must be interpreted correctly. Well- 
designed associative systems result in the generation or production of output whose content, 
location and dynamics makes sense to a user as a meaningful correlate of their own behav- 
iour. 



But this does not mean that only literal or instrumental modalities can be meaningful. Tem- 
poral immediacy permits aesthetic deployment of sleight of hand. In the world, if I knock a 
glass and it falls to the floor splintering, I assume a physical and temporal causality. Assump- 
tion of causality based on temporal order can be designed-in and exploited in interaction 
design. As in film montage, diverse elements and events can be connected by an associative 
or inferential temporal sequencing. The aesthetic manipulation of temporal process is inher- 
ent in interaction. It is worth noting that the verb in 'interaction design' (as opposed to chair 
design or car design) implies process. 
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Lay and Virtuosic Systems 



The distinction between systems designed for untrained public (lay) interaction, and systems 
designed for use by trained interactors is a technical binary which tacitly characterises inter- 
action design in the fine arts. [13] Because art practice is predicated on public exhibition and 
an imperative of some degree of public accessibility, and because interfaces are often un- 
familiar (not a condition experienced in the closed environments of university and corporate 
research labs), the task of providing 'intuitive' access to unfamiliar modalities was (a usually 
unremarked) part of the design task of artists. When interactive digital artwork began, there 
was some focus on celebrating the novelty of digital interaction itself (i.e. mouse-screen co- 
ordination and virtual on-screen 'buttons') while at the same time constructing an easy way 
into such interaction paradigms, usually via deployment of objects, images and structures 
familiar from pre-digital forms. In the early years of interactive art, substantial design effort 
was required to create a context in which an untrained member of the public might be drawn 
into a work which was framed and constrained in such a way that they could be expected to 
do something which wasn't completely 'out of the ballpark', while simultaneously not being 
tediously instructive and didactic. [14] Within a decade, users had thoroughly acculturated to 
the screen/keyboard/pointer, desktop paradigm. Since then, interaction design, in the arts and 
elsewhere, has bifurcated between deploying well known interaction paradigms (which are at 
this point 'intuitive') and the development of novel modalities. In the latter case, artists must 
engage a meta-design task of introducing the user to the special modalities of the work, with- 
out making such introduction itself laborious or instrumental. No one wants to do a tutorial for 
an artwork. [15] 



'Virtuosic systems' comprise an entirely different category of interactive systems as they are 
designed for virtuosic performance by a highly trained 'player'. Such systems are most familiar 
in the context of music and digital musical instrument design. These systems are often highly 
idiosyncratic, predicated on a fused trajectory of player training and system development — 
often the designer/developer is a performer. Such systems re-affirm a conventional binary of 
(active) performer and (passive) audience, whereas systems for 'lay' interaction conflate the 
roles of performer and observer — the performance is the performer observing herself perform. 



Temporality and Poetry in Interaction 



The very existence and success of commercial and commodity interactive digital multimedia 
(and their rhetoric) have, one might suggest, impeded aesthetic progress in the field, because 
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they created confusion between interactivity for instrumental purposes and interactivity for 
cultural purposes. The interactivity of conventional software tools (say a word processor) 
should ideally be 'transparent' and instrumental. What is meant by 'transparent' and 'intui- 
tive' in such discourse is that the behaviour of the system is consistent with previously learned 
bodily realities. A lack of clarity on such issues is typical of the tendency of technical research 
areas to undertheorise. [16] In Heideggerian terms, instrumental software should be 'ready- 
to-hand'. To the extent that it is noticeable, it is bad. This, one might argue, is exactly the 
opposite of what aesthetic interaction ought to be — it should not be predictably instrumental, 
but should generate behaviour which exists in the liminal territory between perceived predict- 
ability and perceived randomness, a zone of surprise, of poetry. 



To the extent that every digital interactive event is analogical, interaction is always poetical, 
and the construction of instrumental systems involves reduction of the poetry quotient. And in 
many cases, the focus of the artist has been precisely to probe the qualities of this analogis- 
ing. This is most often obvious in augmented and mixed reality projects where the behaviours 
in the digitally constructed environment maintain certain consistencies but invert, erase or 
otherwise distort such correspondences. For instance, user representation may be abstracted 
(it is already reduced to two dimensions), but temporal correspondences make it abundantly 
clear what aspects of the image correspond to what body part or gesture. 



Such correspondences leverage deeply embodied understandings of a sensorimotor nature — 
specifically the affiliation of proprioceptive and visual feedback. In a way analogous to the 
example of the Blind Man's Stick as explored by Merleau-Ponty, Gregory Bateson and more 
recently Lambros Malafouris, the real-time computer vision representation of the user almost 
instantly obtains a prosthetic functionality, testifying to the remarkable speed of neurological 
mapping across modalities (Merleau-Ponty, 1965: 245; Bateson, 1972: 434; Malafouris, 2008). 
And indeed, without this, we would not be able to drive a car or use a screwdriver. While real- 
timeness is not easily subtracted from the stick, it was a technical challenge for vision-based 
work, and as such the symptom acquired a name — latency. The very existence of latency — un- 
available in the case of the stick — has led to exploration of the sensorimotor requirements of 
the perception of realtimeness. The persuasiveness, the plausibility of interaction is intimately 
temporal, and indeed, sensory modalities have more or less extended conceptions of 'now'. 
For instance, 'immediacy' (the reciprocal of latency) is much more immediate in hearing than 
in vision. 



The question of interaction has cognitive and phenomenological dimensions which have 
ramifications for the development of adequate aesthetic theory for the practice. In interactive 
work which arises from a tradition of plastic or visual arts, conventional aesthetic language 
imposes an axiomatic subject-object distinction upon the artwork/interactor system. This has 
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the effect of obscuring the very (relational) nature of the experience. A distinction must be 
drawn between two paradigmatic modalities of interaction deployed in cultural practices, 
which we might identify as 'instrumental' and 'enactive'. The instrumental mode, typified by 
HTML links and its hypertextual predecessors (all the way back to HyperCard) regards the 
enaction of a link as simply a way to get from A to B, a connection which ideally is instanta- 
neous and is not marked as an event in itself. A and B are the objects of concern — they are 
objects, and nothing else is of concern. Effects such as fades or wipes, borrowed from video 
and cinematic language, tend to be distractions or at best signify a change of temporal or 
spatial context, register etc. As noted above, Buckminster Fuller's assertion 'I seem to be a 
verb' was itself informed by the doubly 'continuous' nature of the analog electronic signal. 
Both of these conditions are artificially curtailed by the discrete nature of digital data — a 
fact which, via rhetoric of object-oriented programming and the like, may have informed 
the object-centric nature of instrumental interaction. This erasure of temporal process is 
somehow typical of the ways we tend to explain experience — note the cinema 'frame' and 
its metaphoric extension into analog and digital video realms creates the sense that time is 
composed of a sequence of stoppages. 

The lesson of performativity is that the doing of the action by the subject in the context of 
the work is what constitutes the experience of the work. It is less the destination, or chain 
of destinations, and more the temporal process which constitutes the experience. To call 
it 'content' would be again to slip into objectivising language. In what follows I will deploy 
the concept of 'enactive cognition' of Varela, Thompson and Rosch (1992) as it captures the 
ongoing 'structurally coupled' nature of experience (as they say) in 'laying down a path in 
walking' and it is precisely this (performative) aspect of the aesthetics of interaction which 
demands theoretical elaboration. [17] 



Interaction: Embodiment, Gesture and AFFect 



Over the 'heroic' period in question, machine vision for interactive artworks was pursued by 
several artists, with David Rokeby's Very Nervous System being an early case (though Krue- 
ger's Videoplace is the true pioneering work). [18] Very Nervous System, first shown at Ven- 
ice Biennale in 1986, responded to the dynamic of user movement with stereo audio. Rokeby 
notes: 



The installation is a complex but quick feedback loop. The feedback is not simply 'negative' 
or 'positive', inhibitory or reinforcing; the loop is subject to constant transformation as the 
elements, human and computer, change in response to each other. The two interpenetrate. 
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until the notion of control is lost and the relationship becomes encounter and involvement... 
The installation could be described as a sort of instrument that you play with your body but 
that implies a level of control which I am not particularly interested in. I am interested in cre- 
ating a complex and resonant relationship between the interactor and the system. [19] 



The fact that video cameras and real-time video are now a normal part of contemporary 
computers and gaming systems obscures the fact that into the late 90s, machine vision was 
regarded as a non-trivial technical research problem. Getting video into a computer in real 
time required special and expensive peripheral hardware. This makes Rokeby's achievement 
all the more remarkable. VNS ran on an Apple Me, a machine which would have struggled 
to render a single 640x480 ray traced image in 24 hours. He managed with its tiny proces- 
sor to do both real-time machine vision and real-time stereo audio output. He achieved this 
extraordinary result because, in the first instance, his project was concept driven — he knew 
what he wanted out of the technology and he had an adequately deep understanding of the 
analog and digital electronics and coding that he was able to pare away unneeded functions. 
[20] Rokeby's approach was also 'dynamical' and attended to temporal rather than pictorial 
pattern. While a conventional approach would analyse sequential frames pixel by pixel, labo- 
riously identifying and labelling presumed relevant 'objects'; with an artist's education, David 
recognised both that frame-wise thinking was an impediment and that light values on pixels 
do not indicate objects in the world in any unproblematic sense. 



Much of the thinking behind academic and industrial machine vision research still labours 
under the naive conception that frames are a fundamental aspect of reality (rather than a 
skeuemorphic convention) and likewise that 'lines' in images can be unproblematically as- 
sociated with objects or edges in a physical space. Computer science libraries and journals 
are replete with papers on topics like 'edge detection'; in some cases the authors seem quite 
unaware that a video image depends on optics developed in film cameras, and cameras 
themselves were designed to implement the graphical perspective, a conventionalised geo- 
metrical system for representing spatial depth on a plane. [21] 



David was interested in temporal bodily dynamics and recognised that variation through 
time was the data that was critical and pictorial resolution was far less important. Colour 
was therefore dispensable. In fact, David's 1986 'camera' consisted of 64 cadmium sulphide 
light dependent resistors in the back of a wooden box, with a plastic Fresnel lens on the 
front. The resistors, being slow moving devices, effectively damped the system. David in fact 
summed the voltage values for the resistors, to produce one value per time step, and tracked 
the pattern of change in the line generated through time. The parsimonious elegance of this 
solution is characteristic of the technological solutions which technically adept artists of the 
day were compelled to realise due to the double constraint of the condition of the technol- 
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ogy of the day and the limitations of art budgets. VNS is a prime example of what I have 
referred to previously as 'machine parsimony': an ethic of technology design which is elegant 
and economical — at the cost of being application specific. [22] Such an approach is antitheti- 
cal to the conventional commitment to 'generality' and general-purpose tools, which tends 
to be profligate in its use of resources, and encourages a plug'n'play approach to coding, 
which while emulating the look of purist modularity, is antithetical to its spirit. In the context 
of digital cultural practices, the slippage in computer science discourse around this notion of 
generality is most unfortunate. While Turing's mathematical formulation of the 'general pur- 
pose machine' has undeniable value in its context, the notion has oozed weirdly into other 
frames of reference. This has as much to do with marketing rhetoric sleight of hand and with 
economies of scale in the computer industry as it does any principled argument. 



Around the same time as VNS, another Canadian artist (and a teacher of Rokeby) Norman 
White debuted his Helpless Robot (1987). [23] Savagely funny, the Helpless Robot has no 
motive power and its sensor suite is rudimentary. It depends on its verbal persuasiveness to 
entice humans to do its work for it. As a person is drawn into helping the helpless robot, the 
device becomes increasingly impolite and abusive, creating a situation from which the hu- 
miliated human helper must sheepishly escape. Under the hood. Helpless Robot speaks over 
500 phrases organised by emotional categories such as boredom, frustration, arrogance, and 
overstimulation. As such, it is an early example of interactive art practices pre-empting affec- 
tive computing research which began in institutional contexts a decade later. 



In 1990, Luc Courschesne developed Portrait One, an interactive video portrait. [24] The sub- 
ject, a young woman, spoke to the user, who responded by selections from multiple choice 
options on screen via mouse clicks. The system ran under HyperCard on a Mac II which 
serially drove a 12' laserdisc player. Computer and video images were overlaid visually in a 
neatly contrived reflective box — there was no electronic integration of computer and video 
images. By contemporary standards the technology was vestigial and even by the standards 
of its day, it was not particularly technically ambitious. The ambition and the success of Por- 
trait One was in the realm of simulation of affect. While users of far more technically sophis- 
ticated (immersive VR etc.) works came away nonplussed, people came away from Portrait 
One in love. [25] Courschesne achieved with limited means and high aesthetic intelligence 
what others with far more resources were unable to do. His insights into interactive drama- 
turgy, narrative construction and simply, the subtleties of acting, drove Portrait One and later 
projects. The work thus offered a poignant implicit critique of more technophilic enterprises. 
[26] 



Arising from sculptural and installation sensibilities, my own Petit Mai — an Autonomous 
Robotic Artwork (begun in 1992 and first exhibited in 1995) sought to move 'serious' interac- 
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tion off the desktop, out of the shutter-glasses into the physically embodied and social world. 
[27] I undertook the task of building a robust mobile autonomous machine for cultural pur- 
poses. I saw the device, technically, as a vindication of a 'reactive' robotics strategy and a 
critique of conventional Al based robotics, as well as an experiment in artificial sociality. The 
device behaves robustly with the public (albeit in a constrained environment) continuously 
for 10-12 hour days. Socially, it elicits play or dancing behaviour in users. Interaction is driven 
by curiosity and seemingly, a desire to pretend that the thing is cleverer than it is. People 
willingly and quickly adjust their behaviour and pacing to extract as much action from the 
device as possible, motivated entirely by pleasure and curiosity. [28] Petit Mai implements a 
non-instrumental kind of 'play' which is quite incommensurable with conventional computer 
game logic. [29] 



Beyond Pages by Masaki Fujihata (1995) was a work which subtly and exquisitely merged 
the book paradigm with interactive graphics, held together by carefully crafted interaction 
design supported by not particularly adventurous technologies of the time — data projec- 
tion from underneath combined with a tracking stylus for interaction. [30] As one drags the 
stylus across the image of a large book displayed on a projection surface inset into a desk, 
pages flip. The high-resolution sound samples synchronised with the image contrive the 
feeling that these are heavy, course pages. On each page an object is depicted, and stylus 
action moves or changes the object. Propelled by the cursor, a rock shuffles across the page. 
Large crunchy bites are taken out of an apple by an invisible eater. Japanese characters on 
the page are voiced. In an elegant set of visual puns, an image of a light switch on the page 
turns on the (physical) lamp on the desk. A door handle on the page opens the door in the 
video image projected on the adjacent wall. 




Figure 2. Installation view of Beyond Pages 
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As noted above, during this period, artists were (necessarily) at pains to demonstrate the 
new modalities of digital interaction by grafting them on to (representations of) understood 
artefacts and devices. After twenty years, as often as not, the tables are turned. It is the digi- 
tal interface to which users are naturalised and the works serve as a window onto a preced- 
ing technology. It may not be long before users play with Fujihata's Beyond Pages to get a 
sense of what books were like. 



The Implicit, Enactive, PerFormative Body 



Consistent with its interdisciplinary history, the analysis of interactive art has a strongly 
dialectical quality. On one hand, a bone-headedly Luddite approach has all but ignored the 
fact that machine mediated interaction is a novel context, and that without some familiarity 
with the technology, discussion of the work is superficial. On the other hand, technocentric 
approaches tend towards instrumentalisation of the user and the trivialisation of precisely 
the phenomenon which is in need of explication. Ultimately some critical purchase must be 
made upon the behaviour of the complete (user/machine) system. 



This difference might be conceptualised in terms of ascribed modalities of meaning construc- 
tion — is meaning gathered as a result of the extraction of representational tokens, or is it 
enactively constructed in doing? In his insightful work in this area, Nathaniel Stern (2011) 
marshals insights from performance studies to formulate a conception of the 'implicit body', 
a Deleuzian 'body in motion', as an object through which to discuss interaction. Central to 
his analysis is an awareness of the dynamical condition of interaction — a perspective I would 
identify in cognitive terms as 'enactive'. As an interdisciplinary intervention, projects like 
Stern's have the salutary effect of balancing the weight of a technocentric and instrumental 
approach which often has the effect of rendering the user as a 'robot' or a Pavlovian subject, 
capable of a limited range of behaviours elicited by specific stimuli. Crucial to Stern's analy- 
sis is an understanding of temporally and spatially ongoing embodiment as the locus within 
which meaning is created. Such a position is in Agre's (1987) terms 'deictic', and in Picker- 
ing's (1995), 'performative'. This confluence (of Varela and Deleuze, Agre and Pickering — and 
one might add, Evan Thompson, Ezequial Di Paulo, Bruno Latour, Nathaniel Stern, Karen 
Barad and others) points to a new ontological perspective from which interaction, and the 
interactor, can be usefully reformulated. A perspective from which, I would argue, advances in 
interaction design practice might be made, and indeed, in practice have been made, though 
theoretical analysis has been slow to recognise it. 



Fugitive (Penny, 1997) deployed a custom machine vision system for detecting bodily move- 
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ment and large gestures of a user in a 10m circular space. [31] This behaviour drove selec- 
tion of video from a structured database of video and sent it to a motion-controlled video 
projector which displayed the images in varying locations on the wall of the cylindrical room. 
My goals in the interaction scheme of Fugitive were precisely to resist the tendency toward 
scopophilic focus on the image, and rather to draw the attention of the user to the temporal 
continuity of their own embodiment. This was in part motivated by a critique of Virtual Real- 
ity. In conventional VR, the disembodied gaze had the ability to 'move' on preordained paths 
within a pre-structured architectonic environment. 



In Fugitive, the subject is the subject. This presented novel design challenges as one had to 
construct the images and the interaction to counter the normal assumption that when look- 
ing at an image, it is the image, rather than the looking, which is important. I made explicit 
at the time, in Fugitive, the continuity which structured the experience was the subjective 
temporal continuity of the users' embodiment, or more correctly their kinaesthetic aware- 
ness. Virtual 'worlds' arose and collapsed on the basis of that continuity. 



One way of thinking about interactive artworks is that they might provide a context in which 
engagement with the work constructs a condition which requires further action in order to be 
resolved; in which artefacts and effects are arrayed spatially and temporally in such a way 
as to encourage the formulation of novel ideas. The temporality of the process is unavoid- 
able, and its design constitutes a kind of synthetic enactionism. The arrangement of such 
artefacts and effects in a way that optimally stimulates such processes (bearing in mind 
questions of demographics and cultures) is then, the cognitive dimension of the task of inter- 
action design for aesthetic purposes. 



Terminal Time and Al-based Art 



In the later 90s an eminently interdisciplinary grouping — Steffi Domike (a documentary film- 
maker) Paul Vanouse (a media artist) and Michael Mateas (an Al specialist) — produced a 
project which, while only marginally 'interactive' and not exactly adapting to a user in real 
time, deployed relatively sophisticated Al techniques to build documentary films 'on de- 
mand' and according to the political profile of the particular audience. [32] In Terminal Time 
the audience was polled in multiple choice form on its positions on various issues regarding 
technology, religion, science, politics, gender issues (etc.) their response gathered by a sim- 
ple applause meter (microphone). Based on this data, the system assembled a documentary 
on the history human civilisation, combined from hundreds of music and film clips and short 
voice-overs, which was then presented to the audience. Terminal Time stands as a witty 
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commentary on contemporary political media — the audience gets the history it deserves. 
Terminal Time generates as output, a formally conventional linear narrative, it trades in a 
currency of symbolic tokens, it addresses its multiple audience as a demographic mass and 
has no concern with the ongoing embodied behaviour of it audience. 



As experiences, conventional interactive systems tend to fall into two camps — ones that are 
simple and therefore easy to understand, and which therefore either become familiar and 
automatic or boring, and others which are so complex as to be baffling and cause all but 
the most dedicated to fatigue. By 'conventional' I mean that the interactions themselves are 
hardcoded. This is a kind of meta-stability. To move beyond this stalemate, in 1997, I pro- 
posed the idea of a dynamical 'auto-pedagogic interface' — an interface which observed and 
learned from its user(s) how well they understood the system, and ramped up complexity 
gradually as a result of constant monitoring. [33] In this context. Terminal Time represents a 
phase shift in the sophistication of strategies of database combinatorics in artworks. To that 
date, interactive works had usually taken recourse to only the most simplistic of selection 
procedures (lookup tables, pseudo-random choices). Terminal Time deployed a wide palette 
of sophisticated Al techniques to dynamically construct the work for every iteration. This 
strategy was based in Mateas' reflexive view of Al. Consistent with the critique of the time, 
he rejected the larger project of Al regarding simulation of human intelligence. He insightfully 
observed that in the context of that discipline, some very intelligent people had developed 
sophisticated tools for automated reasoning and such tools were ripe for application in inter- 
active art. Mateas (2001) refers to this practice as 'Expressive Al'. 



Phenomenologically-based critiques of Al, the elucidation of the frame problem (and the 
related symbol-grounding problem) and the more recent critiques by Agre have themselves 
'framed' the domain in which (good old-fashioned) Al can be effective — a domain of logico- 
mathematical manipulation of symbolic tokens. Philip Agre is so eloquent on this subject that 
a full quotation is justified (but only on the left): 



...the privileged status of mathematical entities in the study of cognition was already central 
to Descartes' theory, and for much the same reason: a theory of cognition based on formal 
reason works best with objects of cognition whose attributes and relationships can be com- 
pletely characterized in formal terms. Just as Descartes felt that he possessed clear and dis- 
tinct knowledge of geometric shapes, Newell and Simon's programs suffered no epistemo- 
logical gaps or crises in reasoning about the mathematical entities in their domains (1997: 
143). [34] 
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A! foundered on extraction of such tokens from the world and the re-connection of the result 
of such processing with the world. Indeed, even to speak of connection with the world as 
constituted by input and output is to subscribe the worldview which has been found to be 
defective. Thus the useful applicability of Al techniques to interactive art and its challenges, 
as it has been framed here, is limited. Interactive art captures and epitomises many of the 
contexts in which Al has been troubled — embodied engagement with an explicitly open-end- 
ed world of multivalenced cultural properties. Interactive art is the New York of Al — if it can 
make it there it can make it anywhere. 



There is a domain of cultural practice where the constraints on Al techniques are not an 
impediment. On the net, the applied fields of data-mining and machine-learning have dem- 
onstrated significant successes. The popular name for autonomous agents in game-worlds 
is 'Als'. It is worth noting why this is. It is because on the net, the processing of the mate- 
rial world into bits has already been done (by people). Everything on the net is pre-digested 
for CPUs into a numerical value, one just has to find and get it. This relieves such systems 
of much of the tricky (far more tricky than Al rhetoricians admitted) task of interpretation of 
sensor data derived from the electro-physical world. 



Giver of Names (David Rokeby, 1998) presents as novel a reading of artificial intelligence 
(and especially the area of 'natural language' analysis and generation) as VNS did of ma- 
chine vision. [35] Giver of Names speaks cryptic sentences produced by a metaphorically- 
linked associative connectionist database of objects, ideas and sensations. The system is 
seeded by shape and surface recognition information derived from a machine vision sys- 
tem which views (arbitrary) objects placed on a pedestal by visitors. These utterances are 
fed back in to the system, generating a stream of solipsistic musings which by turns sound 
poetic, philosophical or like the ravings of an obsessing schizophrenic. Here the preoccupa- 
tion is not with interactivity, but with the behaviour of a solipsistic self-reflexive autonomous 
system. 



Artificial Life, Autonomous Agents and Virtual Ecologies 



Another work which inventively addressed concerns of the period is El bal de Fanalet/ Light- 
pools, by Narcis Pares, Roc Pares and Perry Hoberman (1998). [36] El bal de Fanalet/ Light- 
pools combined multiple spatial tracking and interactive artificial life-based graphics with 
artefacts derived from Catalan popular culture. [37] El bal de Fanalet/ Lightpools captures 
three important themes of 90's interactive art. The first was the articulation of the notion of 
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Figure 3. El Bal De Fanalet/Lightpools showing 'lightpools' and 'falanets' 



'the virtual' which was a central term in the discourses of the period, and the exploration of 
embodied interaction with the 'virtual'. The second was an exploration of 'artificial life', the 
possibility of implementation of life-like forms and behaviours in computational systems. A 
third concern for artists working with interactive systems was the limitation of conventional 
computer platforms and sensor systems to single user application. There was much interest 
in devising viable multi-user systems which attend to the small and large group social dy- 
namics so important in human interaction. A set of more practical concerns involved how to 
instrument members of the public so such presentations were viable in public spaces. Exten- 
sive 'suiting-up' was labour intensive and time consuming and required complex logistics and 
support. El bal de Fanalet/ Lightpools addressed all these concerns with some elegance. 



Pares, Pares and Hoberman implemented a world where users could grow virtual organ- 
isms which would then interact with users and other entities in the 'virtual' realm. Aspects of 
these modalities were commonly addressed by other artists. Char Davies and Sommerer and 
Mignonneau come to mind. But El bal de Fanalet/ Lightpools is particularly successful in ad- 
dressing and resolving a combination of these issues. 



Users wielded a fanalet, an interface device modelled on the eponymous paper lantern 
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used in a traditional Catalan dance festivity. The signification of the lantern was deployed 
metaphorically to conceptually join the physical world inhabited by the users with the vir- 
tual world 'under the floor'. Here three (at least) separate conceptual realms are joined — the 
world of the fanalet and the dance, the world of fish etc. in ponds, and the realm of artificial 
life creatures. The 'glue' that held these metaphors together was real time integration of 
user gesture with projected content. Tight gestural correspondence and control of latency 
gave actions an immediate sensorimotor veracity. The 'lamp' interface device contained a 
spatial tracker and the tracker localised the area where light would shine from an overhead 
projector. The virtual realm was 'exposed' in the 'lightpools' associated with each fanalet, 
in the way that at night a lantern might illuminate an area of a lake or pond. Movement of 
the fanalet by the user encouraged movement of the virtual creatures, and new behaviours 
emerge when users merge their lightpools. Here is a case example of complex interactive 
metaphors being intelligible by virtue of their relation to real world scenarios and dynamics. 
The question of how these metaphors can be stretched while remaining coherent is a rich 
area for cognitive-linguistic research in the field. [38] 



Works such as El bal de Fanalet and Petit Mai reflect a general interest among the commu- 
nity at the time in Artificial Life. Alife marked a radical swing in computation and Al circles 
from symbolic approaches to systems modelled on biology, self-organising and dynamical 
systems and emergent complex order. Colonial animals — slime molds, sponges, termites, 
ants and bees — became the poster-children for emergent complex behaviour from simple 
elements. Cellular automata were seen to model such behaviour computationally. Evolution- 
ary, genetic and ecological metaphors and methods (Holland, 1975; Ray, 1992), neo-neural- 
net connectionism and Darwinian computational environments (genetic algorithms) dem- 
onstrated significant success (Sims), as did bottom-up and reactive robotics (Brooks, 1991; 
Steels, 1996). In the arts, there was a remarkable flowering of new practices based in these 
ideas (Penny, 2010). 



My own work Sympathetic Sentience (Penny/Schulte 1996) was inspired by such develop- 
ments. [39] My goal was to build a minimal physically instantiated system of multiple com- 
municating units which demonstrated emergent complex behaviour. Sympathetic Sentience 
was constituted by twelve independent, more or less identical, wall mounted sound produc- 
ing devices. These communicated in a serial loop via custom infra-red hardware. Alone, 
each unit could emit only an occasional chirp, but in a loop, each device voiced a constantly 
changing but acceptably ordered rhythm and melody. The system on its own generated com- 
plex and constantly changing output and was self-stabilising, it possessed a native capac- 
ity for quasi-stability — after initial build-up, the chirp/silence ratio stabilised but fluctuated 
around 50%, it never went silent or fully saturated. (In cybernetic terms it was a homeostat 
of sorts). The system achieved this result with little but a simple instable pulse generator, 
an XOR gate and a short time delay in each unit. The system did have minimal, suppres- 
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Figure 4. A user interacting with Sniff in a storefront 
installation of the work 



sive 'interactivity' — a user could unknowingly interrupt an IR beam and thus impose a gap or 
silence in the melodic passage, which would then slowly fill up again. The focal concern was 
not with human agency, but with the agency, autonomy and creativity of a minimally com- 
plex system. 



A more recent work which combines emotion modelling with embodied interaction is Sniff by 
Korolina Sobecka and Jim George (2009). [40] Sniff is member of the Alife genus of on-screen 
interactive semi-autonomous agents which must count Weizenbaum's Eliza as their ances- 
tor. Sniff is a lively pup, or rather, a behavioural portrait of a lively pup rendered on a large 
back-projection screen. Sniff interacts with visitors via machine-vision based sensing. Sniff is 
rather finely physiologically modelled and 3D animated, but in a deft artistic touch. Sniff es- 
chews the baroquity of texture maps and synthetic fur and is presented as a fine-resolution 
polygonal model in white lines, in a featureless black space. The contrast between the veri- 
similitude of Sniff's behaviour and the abstraction of its visual representation heightens the 
persuasiveness of its physiological and behavioural modelling. Sniff is a rather subtle canine 
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behaviour-portrait, it will identify with one of several visitors a primary interactor, and has a 
sophisticated behavioural memory. 

Performative Ecologies by Ruairi Glynn (2008) is a more abstractly mechanical work which 
attempts to realise embodied agents which learn from their environment and improve their 
behaviour. It consists of three identical, exhibitionistic and somewhat insecure motorised 
sculptures which dance for their supper, as it were. Each has a tail part — a motorised illumi- 
nated rod which it can turn and light up variously, and a head part, an infra-red video cam- 
era on a motorised neck. When functioning as designed, each unit identifies a visitor using 
face recognition software, and performs a dance for her with its tail, all the while monitoring 
the attentiveness of the visitor, using gaze tracking. The longer the visitor attends, the better 
the dance scores. Later, using genetic algorithms, these devices develop new dances from 
the best scoring previous dances, to be tried out on the next day's visitors. In a limited way, 
these devices move beyond the 'look-up table' paradigm of interaction where all behavioural 
possibilities are pre-scripted, to a mode which can truly produce novelty or surprise. [41 

The Role oF Epistemic or PerFormative Action in Interaction 



A work of very different scale and historical reach is Rafael Lozano Hemmer's Displaced 
Emperors (1997), a part of his 'relational architecture' series of works. [42] The projection 
installation took place on the multi-storey exterior of a Hapsburg villa in Linz, Austria. A user, 
equipped with a custom triangulating sonar tracker on the back of her hand, makes a wav- 
ing gesture in the direction of the castle, and the image of a giant hand moved across the 
fagade. Using synchronised image projections, this waving appeared to wipe the exterior 
facade of the building away, revealing period interior rooms. These room views were in fact 
taken in the Chapultepec Palace in Mexico City, the residence of the short-lived Mexican 
Hapsburg dynasty. The waving/wiping gesture collapsed one physical location onto another, 
and directly — via the immediacy of sensori-motricity — induced reflection of the historical pe- 
culiarities of the colonial period. 



We should make a distinction between interface and interaction modalities which are de- 
ployed as a mechanism for exploring 'content', and modalities which themselves contribute 
to the accumulated meaning or experience of the work. As discussed above, in some interac- 
tive work, interactive modalities are taken as transparent and given: the dynamics of interac- 
tion were conceived as a means to an end which was primarily found in the 'content' of the 
work (as if interaction dynamics were not always part of the 'performative' content). In other 
cases, such as Displaced Emperors, the dynamics of interaction play a key role in the overall 
construction of meaning. We might suggest that content-centric interactives are retrogres- 
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sive, and simply articulate the representational idiom of painting, film and video, while works 
in which the dynamics of interaction are themselves the 'content' of the work occupy a more 
progressive position. 



The recognition that the process of interaction is both embodied and quintessentially per- 
formative provides a position from which to build out an aesthetic theory. Andrew Picker- 
ing's (1995) formulation that the representational and performative idioms are distinct and 
perhaps incommensurable is relevant. Originally applied to questions of scientific knowledge 
and practice, this binary is ripe for application to arts practices, for conventional plastic 
arts artefacts are representational artefacts par-excellence. I want to assert that interactive 
cultural practices, while deploying representational components, prescribe a performative 
ontology — some more than others. Again — to the extent that the mechanisms of interac- 
tion are naturalised, automatic, 'intuitive', ready-to-hand — they do not play a significant part 
in the epistemological circuit of the work. But to the extent that I have to bend this way, 
climb that ladder, or stand with my feet in cold water, the doing of the work, the embodied 
and performative dimensions are and must be designed as (often major) components in the 
overall meaning of the work. Extensive work remains to be done in assessing how interac- 
tions mean. Here the distributed cognition research of Edwin Hutchins (2006) and David Kirsh 
and Paul Maglio (1994) offer ways in to this inquiry, as both researchers are concerned with 
how bodily action plays a role in cognitive processes involving manipulation of artefacts and 
exploitation of images. 



Building such meaning has several aspects. Making a user complicit in the construction of an 
unfolding experience is a powerful technique for establishing engagement and commitment. 
If seeing a video in which a person is pushed off a balcony is disturbing, pushing them your- 
self — or being put in the position of choosing to — is an order of magnitude more so. (Unless 
of course such action is routinely trivialised, as in first person shooter games). Aside from 
ethical issues, basic sensorimotor realities anchor the actions of interaction in a way that 
dissociated contemplative vision can never achieve — if indeed it were possible. As embodied 
beings, we conduct our path through the world in the form of sensorimotor circuits which 
have no beginning or end. Contrary to received wisdom, humans rarely perceive then act, 
but understand the world in a synaesthetic and proprioceptive fusion of sensing and action, 
often acting to perceive the action to calibrate the action. An aesthetic theory of interaction, 
then, must include a choreographic understanding of user action. 



It is not just doing, not just the awareness of doing, but the cognitive dimensions of doing 
which are important dimensions of interaction design. Kirsh and Maglio (1994) demonstrat- 
ed a phenomenon they call 'epistemic action'. Epistemic action describes the gathering of 
knowledge through action in the world. That is, certain kind of things can only be thought, or 



fibreculturejournal.org FCJ-132 95 



Towards a Performative Aesthetics of Interactivity 



can be thought faster, more efficiently or more richly, by the manipulation of objects in the 
material world. Doing math on paper is an example which trades in symbolic notional sys- 
tems, moving the scrabble letters about to suggest words is somewhat more materialised. 
An array of utensils and ingredients on a kitchen bench might suggest action, the shape of 
the knife facilitates certain kinds of actions (and not others), but does the arrangement and 
design of implements support mental computation? Kirsh and Maglio's Tetris research ad- 
dresses time constrained problem-solving. The challenge of applying such models to inter- 
active art practice is that only the most tedious kinds of interactive art, and only the most 
tedious approaches to interactive art, treat it as a puzzle-solving task. In more exemplary 
cases, the conditions are set for an inquiry whose outcome, while framed, is open-ended 
and designed to be generative, allowing the possibility of hypothesis formulation, rather 
than simply resolution. Cognitive science, with its quantitative scientific roots, has under- 
standably shied away from such questions, seeing them as out-of-ballpark. This is one of 
the ways in which scientifically based practices find unresolvable discontinuities in the arts. 



Similarly, I have several times been asked by HCI professionals if I have undertaken user- 
studies of my works, and the question has always seemed absurd. The idea that the 'out- 
come' of the experience is not proscribed is fundamental to the rationale of the practice. 
It would be an amusing dinner party game to design such a questionnaire. 'On a scale of 
1-10, rate your surprise' ... 'How satisfactorily did the work challenge your unquestioned as- 
sumptions?'... 'What did it make you think about that you had not expected to think about?' 
Would it be possible to analyse epistemic action in the domain of interactive art, where 
the task might be formulated not so much as getting the right answer, but asking the right 
question? 



Becoming Becoming 



Through this essay, I have argued in various ways for the relevance of the processual in 
thinking about interaction. I noted that such sensibilities underlay much 60s art, and played 
a role in the design discourse of that techno-guru of the period Buckminster Fuller. Here 
again we might observe the prescient but inchoate way these ideas come up first in the 
arts. Yet how can a felt embodied idea be anything but inchoate, without the development 
of a discursive skin? In their day, works like Senster and Videoplace could not be appreci- 
ated for want of a discursive context, so likewise it has been with 'process' in general. 



In the years since, such a discursive skin has indeed begun to surface, in patches as it were. 
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In Artificial Intelligence, Agre and Chapman (1987) proposed Deictic programming which 
contested the objectivist 'gods eye view' assumptions of conventional Al. Situated and reac- 
tive robotics (Brooks, 1991) were similarly inclined as were discourses of 'emergence' and 
iterative processes in Artificial Life, Self Organisation and Dynamical Systems Theory (Kelso, 
1995). In cognitive science, Enactivism, situated and distributed cognition challenges cogni- 
tivist attitudes. [43] In the humanities, the rise of performance theory (in some forms) con- 
tests the excesses of the linguistic turn. In science studies. Actor Network Theory and Picker- 
ing's Mangle both destabilise conventional objectivist science discourses. In art theory, we 
saw the rise of relational aesthetics, and in philosophy, a resurgence of interest in Spinoza 
and Bergson, brokered by Deleuze (2001) and later Massumi (2002), and a resurgence of 
pragmatism (James, 1907; Dewey, 2005). This is quite a patchwork, and some of the pieces 
might feel a little strained. But across diverse fields in the late twentieth century, such ap- 
proaches have lurked on the margins of positivist disciplinary discourses. To go beyond a call 
for the recognition of the fundamentally embodied and distributed nature of cognition, the 
upwelling of arguments (in such diverse fields as feminisms, performance studies, science 
studies, and cognitive and neurosciences) which contest presumed axiomatic binaries of sub- 
ject/object, world/representation suggests a fundamental and large scale change in ontology 
which one might be excused for identifying as a paradigm shift. 



Contemporary cognitive science, informed in part by Heidegger, Husserl and Merleau-Ponty, 
has brought into question not only cognitivist representationalism, but the fundamentally 
Cartesian binaries of mind/body, self/world and subject/object. The deep ontological ramifi- 
cations of all this is captured with some precision by Karen Barad (2007), in which she argues 
that the very construction of subject and object are historically contingent, and proceeds 
to propose a radically materialist and performative ontology which sees 'phenomena' as 
primary and subjects and objects contingently forming or falling out of a process of 'intra- 
action'. Such an approach would be consistent with the performative ontology of Pickering, 
the 'laying down a path in walking' of Enactivism and the 'ongoingness' of O'Reagan and 
Noe (2001). This ontological reformulation has direct relevance to the theorisation of, and the 
creation of, interactive artefacts. 



In Rafael Lozano Hemmer's Displaced Emperors the gesture of the swipe of an outstretched 
hand wipes the fagade off the palace, revealing (images of) the rooms inside. The fagade, 
in both a real and metaphorical sense, is destroyed, performatively implicating the user in a 
violent act. The user, standing in front of the imposing edifice, makes a grand gesture. They 
could make a wriggle or a flick, but they tend to make an expansive full arm gesture. How 
such a style of gesture is implied or specified is a mysterious aspect of the design of the sce- 
nario — but it happens. And the value of that gesture — grand and empowered, is an integral 
part of the experience. Perhaps the gesture is implied by the immediately prosthetic sensori- 
motor linkage of the gesture with an image of a (giant) hand, which strokes the surface of 
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the fagade as a shadow would. The embodied sensorimotor understanding of the movement 
of my shadow as action at a distance is exploited here. The strength and scale of the arm 
is amplified, as with a sword or a staff, and this powerful force-amplified prosthetic arm 
wipes the masonry of the fagade away — at least it appears that way, sans the noise and 
rubble and dust. Here then, embodied and gestural engagement constitutes something like 
epistemic action — but in the process, nothing is resolved. Through enactive or performative 
action the user is immersed in a context loaded with the paradoxes of colonialism. 



Autistic and Solipsistic iviaclnines 



In recent years a new class of devices has arisen which offer a perversely witty response to 
the preoccupation with the user. These works possess complex behaviour and are so autono- 
mous as to have no connection with human viewers. In such works, the visitor engages in a 
conventional mode of passive observation of an autonomous machine negotiating its envi- 
ronment. The biological or ecological implications suggest a kind of synthetic ethology. Two 
recent works exhibit a resurgence of a minimalist/formalist sculptural aesthetic. They also 
have in common a commitment to materiality and to integrating electro-physical realities 
as part of the larger computational a system. That is, the elegance of their formalism has 
extended into the machinic/performative dimension of their existence. 



Der Zermesser by Leonhard Peschta (2007) is a large but minimal geometrical robot which 
engages in a sensitive relation with architectural space, expressing its response as pertur- 
bations of its tetrahedral form. [44] One might call it 'architaxis'. Each node is instrumented 
and motorised such that it can both move and adjust the lengths of is vertices. Der Zermess- 
er works on a slow, non-human timescale, like a mollusc on a rock. This work is exemplary 
of a long-delayed reunion of aesthetic systems of conventional plastic arts with interactive 
art. The geometric minimalism of Der Zermesser recalls formalist sculpture, that epitome of 
modernism, and indeed, it could be said to implement formalist aesthetics as a machine, 
in the same way that computer hardware implements Boolean algebra. And certainly the 
formalisms of coding and computer engineering do have immediate sympathies with the 
modes and methods of formalist sculpture. The formal coherence and elegance of Der 
Zermesser is carried through to its electronic aspects — the geometrical nodes are also mo- 
tor, sensory and computational nodes. The tetrahedron, the first platonic solid, the strongest 
polyhedron, is an icon of engineering efficiency, and is also manifested in a myriad of bio- 
logical forms, evidence of the effectiveness of evolutionary design. Der Zermesser captures 
this duplicity as an adaptive geometry. 
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The Conversation, by Rolf Baeker (2010) is a self-referential and homeostatic colonial 
machine — a model of a colonial organism rendered in electro-physical machines. [45] The 
Conversation is a closed-loop computational system combining electro-mechanical, analog 
and digital electronic components, presented in an elegant and minimal sculptural form — a 
ring made up of a 99 solenoids. The solenoids pull on wires connected to three concentric 
rubber rings in the centre of the device. With a machine parsimony based in an understand- 
ing of electro-physical phenomena, Baeker monitors fluctuations in the magnetic pull of the 
solenoids via their current consumption (where others might have added an array of sen- 
sors). A floating metal ring — a platonic apparition, paradoxically pulsates with the vibrancy 
of the living. If the metal ring is the cell wall, the three fluctuating red rubber rings define the 
nucleus, with radiating strings relaying tensions back and forth between the two. Crucially, 
the physical system is not 'driven' by code, but in dynamical and cybernetic sensibility, the 
moment by moment condition of the material array drives resonating cycles of oscillations 
and dampings within the system. 



Conclusion 



The general historical picture I hope to have drawn here places the development of an 
aesthetics of interaction within a developing technological context. During the 'heroic pe- 
riod' of interactive art, big questions such as: 'how can we deploy computational capability 
in artworks?' and 'how can we integrate computation with material, sensorially immediate 
practices?' motivated work. The technological challenges of interaction, like the other tech- 
nical challenges of the 90s — problems of wireless communication, computer graphics and of 
machine vision — are now effectively resolved. Culturally, the novelty of the scenario of the 
machine which responds to a user in real-time has clearly worn off. In digital cultural prac- 
tices, exploration of the modalities of interaction has been fairly thorough, though there is 
always room for inventive exploration of the subtle complexities of the poetics or aesthetics 
of interaction. In my opinion, future development of the aesthetics of interaction might use- 
fully be framed as three areas of concern: the material artefact, the code/machine system, 
and the dynamics of interaction. 



For two decades, the computational has been more or less 'pasted on' to artefacts and so- 
cial structures. And with the technology, the rhetoric of cognitivist computation has also usu- 
ally been more or less uncritically pasted on. Works like Giver of Names, Der Zermesser, and 
The Conversation, demonstrate an integration of sculptural and embodied dimensions with 
a sensorial and performative orientation to coding, into a coherent aesthetic and theoretical 
approach (we heave a sigh of relief and whisper 'at last!'). In their integration of materiality 
and computation, these works reject a dualist computationalist separation of software and 
hardware, information and matter, control and action. For me this signals a new maturity in 
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the cultural and theoretical grounding of work of this kind, where 'interactivity' is subsumed 
into a wider field of autonomous machine behaviour. As I have proposed elsewhere, in our 
current era of ubiquitous computation, the universe of live data which was once called 'the 
virtual' is increasingly anchored into physical and social context via a diversity of digital com- 
modities. The technologies, techno-social structures and modalities of interaction which per- 
mit this (re)union were workshopped and prototyped in 'media arts' research and elsewhere 
over the past quarter century. 



A second generation of practitioners naturalised to the digital are pursuing a more organic 
interrelation between machine behaviour and sensoriality/materiality than their forbears 
could, in part due to technological developments but also due to a maturation of the fine arts 
context. Meantime, an increasingly code and hardware-literate community of artists are able 
to deploy more sophisticated aesthetic code-machines. A deepening and theoretically sub- 
stantiated conception of interaction and enactive cognitive process inhering in a performative 
ontology promises more rich and subtle systems. And lastly, the conception of 'interaction' 
has been expanded beyond user-machine, to larger ideas of behaviour between machines 
and machine systems, and between machine systems and the world. This leads to a kind of 
machinic ecology, and potential useful application of actor network theory. 



I have proposed that across a diverse range of disciplines, we are on the cusp of a veritable 
Kuhnian paradigm shift toward a performative ontology. Such moments are, in the terms of 
Hakim Bey and Peter Lambourne Wilson, truly 'temporary autonomous zones' (Bey, 1991) 
and open up temporarily at moment of flux. Like all good technical ideas, it has a heady 
whiff of metaphysical liberation about its tattered edges. Over the last fifty years or so, such 
shifts have appeared (on the horizon so to speak) only — mirage-like — to evanesce. For exam- 
ple, it is salutary to note how aspects of the cybernetics which could be recuperated to the 
positivist establishment lived on, control theory being a case in point. In any case, in such a 
Kuhnian shift, the intractable is rendered trivial by an orthogonalising shift of perspective. In 
my opinion, the practice and performance of interactive art itself is an integral part of that 
ontological shift, and that shift offers leverage on theoretical questions which have seemed 
vexing under previous theoretical approaches. 



Biopgraphical Note. 



Simon Penny is Professor of Arts and Engineering at University of California Irvine. His prac- 
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institution building in Digital Cultural Practices, Embodied Interaction, and Interactive Art. He 
makes interactive and robotic installations utilising novel sensor arrays and custom machine 
vision systems which address critical issues arising around enactive and embodied interac- 
tion, informed by traditions of practice in the arts including sculpture, video-art, installation, 
and performance; and by ethology, cognitive science, phenomenology, human-computer in- 
teraction, ubiquitous computing, robotics, critical theory, cultural studies, media studies, and 
Science and Technology Studies. 



Notes 



[1] I have explored related themes in two recent papers: 'Trying to be Calm,' and 'Desire for Virtual 
Space' (2012, 201 1). The current paper addresses related issues across the same time period. As 
such the three might be usefully read together. 



[2] Discussion of these works can be found at www.simonpenny.net (works). 



[3] This technological obsolescence is a profound impediment to pedagogy and historical research. 
It is a regrettable fact that much contemporary media art history and theory discusses works 
which the authors themselves have not actually experienced, and often relies on poor documenta- 
tion and/or other textual accounts. This situation would be regarded as intolerably unscholarly in 
conventional art history. Imagine the contempt an art historian would be held in if they presumed 
to write on, say, the ceiling of the Sistine chapel without having actually seen it. So much worse 
then, is relying on photographs (!) of a work which was originally dynamic and multi-media. 



[4] Eigenwelt der Apparatwelt, Pioneers of Electronic Art, Ars Electronica 1992 exhibition cata- 
logue, 1992. 



[5] See http://www.senster.com/ 



[6] See http://www.youtube.com/watch?v=dqZyZrN3PI0 



[7] See http://www.youtu be.com/watch ?v= I LULRImXkKo (Quaint BBC coverage) 
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[8] Dietrich Prinz wrote the first chess playing program for a general purpose computer. The pro- 
gram first ran in November 1951 on the Manchester Ferranti. Prinz was the head of programming 
at Ferranti. 



[9] Steve Wozniak recently commented when asked if, in the late 80s, he foresaw internet porn, 
that the early Apples did not have enough memory to render a (colour) image, to say nothing of 
real-time video streams. (Radio interview on 'Wait, wait, don't tell me', December 2010, personal 
notes) 



[10] See my forthcoming essay: 'Emergence, Improvisation, Interaction', Lewis and Piekut (eds) 
Oxford Handbook of Improvisation Studies. 



[11] Likewise, sophisticated machine vision procedures, such as 'body-segmentation' — the nerdy 
moniker for inferring skeletal anatomy from single or multiple images — which were non trivial and 
cutting-edge technical problems circa 2005, are now part of commodity devices, i.e. the Kinect. 



[12] As far as I am aware, this assertion remains speculative. Given the extensive evidence in 
developmental neurology for similar neurological adaptation, I am persuaded by this idea. As we 
move into the generations of those 'born digital' there is a call for neurophysiological research into 
structural and behavioural differences between those naturalised to digitality and those who came 
to digital systems later in life. 



[13] As indicated, I am here focusing my attention upon interaction design in the fine arts, and 
referring to practices which extend beyond the desktop or hand held device. I am well aware of the 
rich complexities of the domain specific interaction modalities in networked interactions from vir- 
tual communities (Second Life) to massively multi user gaming (World of Warcraft) to dimensions 
of social media from Facebook to Twitter, but these are beyond the scope of the current text. 



[14] Stories of such unfamiliarity from the early 90s are common, such as that of the young woman 
driven to tears by the computer mouse. She had reasoned, perfectly reasonably, that the mouse 
should 'face' the screen, i.e. with the 'tail' toward her, which inverted all her actions with respect to 
cursor movement. 



[15] I use the term 'user' here (grudgingly) and in what follows to refer to anyone interacting with 
an interactive artwork. Many including myself have deplored (the instrumentality of) the term, and 
have sought alternatives and coined neologisms, but alternatives have been clunky and have not 
achieved general acceptance. 
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[16] See Agre (1997). A more subtle point is that to the extent that such modalities are intuitive 
in this sense, neurological impact will be minimal. But as digital behaviours themselves become 
'naturalised' at the level of culture and of embodied cognition, we will see a drift in what is taken 
to be 'intuitive'. 



[17] This phrase is the title of chapter 11 of The Embodied Mind; Cognitive Science and Human 
Experience by Francisco J. Varela, Evan Thompson, Eleanor Rosch (MIT press, 1992) The authors 
note 'Our guiding metaphor is that a path exists only in walking...' (241). The term has been taken 
up in aspects of contemporary cognitive science and Philosophy of mind, notably the work of 
Kevin O'Regan and Alva Noe (2001). 



[18] See http://www.youtube.com/watch?v=GALMmVZ49Pc 



[19] See http://homepage.mac.com/davidrokeby/vns.html 



[20] In computer science and engineering circles, the idea of generality or general applicability is 
a cardinal virtue. This ethos can lead to systems which do many things badly and nothing well. 
The approach of many artists (including Rokeby) is to achieve a specific goal with quality and 
elegance. An approach which is fundamentally pataphysical. 'Pataphysics will be, above all, the 
science of the particular, despite the common opinion that the only science is that of the general'. 
Alfred Jarry 



[21] The geometrical basis of perspective was laid by the Arab mathematician Alhazen (al-Hasan 
Ibn al-Haytham) in 1021. There is nothing 'natural' about perspective, it is not the 'true' way to 
represent the world, and it is not an integral biological aspect of human vision. 



[22] I developed the concept of 'machine parsimony' in the mid 90s to describe this kind of 'appli- 
cation specific' development which did not subscribe to the requirement for generality. 



[23] See http://www.year01.com/helpless/statement.html 



[24] See http://www.medienkunstnetz.de/works/portrait-one/ 



[25] I witnessed this at SIGGRAPH91 in Las Vegas. 
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[26] It is no accident that these three pioneering works all arose in Canada. Canada was remark- 
ably proactive in funding and otherwise supporting 'media-art' research in the period and in the 
small international community of interactive art — Canadian artists (etc) were ubiquitous and rec- 
ognised as doing generally progressive advanced work. 



[27] See http://www.simonpenny.net (works — petit mal). 



[28] Interestingly, the only demographic who were unwilling to interact were adolescents. 



[29] The conception of 'play' as encapsulated in typical computer games is a decidedly narrow 
slice of modes of play and has an instrumental cast to it. When, for instance, the paradigmatic 
'first person shooter' is stripped of its colourful monsters and futuristic weapons, computer game- 
play resembles the worst qualities of industrialised labour: constrained and highly repetitive tasks 
executed in social isolation, a tight harnessing of user and machine, rewards linked to high rates of 
production, to say nothing of the covert inculcation of military skills. 



[30] See http://www.iamas.ac.jp/interaction/i97/artist_Fujihata.html, http://www.youtube.com/ 
watch?v=6Ek2DW7aV68 



[31] See http://www.simonpenny.net/works/fugitive.html, http://www.simonpenny.net/works/fugi- 
tive2.html 



[32] Terminal Time http://www.terminaltime.com/ 



[33] See my 'Agents as Artworks and Agent Design as Artistic Practice' (1999). This kind of adap- 
tive technology harks back to cybernetic precepts but has rarely been pursued in artwork. A work 
which has pursued this to some extent is Performative Ecologies by Ruari Glynn (2008) (discussed 
below). 



[34] Agre's remarkable paper is the most insightful inquiry into the culture of Al I know of. 



[35] See http://homepage.mac.com/davidrokeby/gon.html 
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[36] See http://www.galeriavirtual.org/varis/lightpools.ogg 



[37] El bal de Fanalet/ Lightpools used the GAMS triangulating sonar sensor developed by Will 
Bauer and Rafael Lozano-Hemmer, also deployed in Displaced Emperors (Lozano-Hemmer, 1997). 



[38] See for instance the work on concept blending by and Fauconnier and Turner (2008). 



[39] See http://www.simonpenny.net/works/sympathetic.html, http://www.simonpenny.net/works/ 
sympatheticll.html 



[40] See http://vimeo.com/6400266 



[41] Sniff and Performative Ecologies were included in Emergence, a show of Artificial Life Art cu- 
rated by Simon Penny and David Familian at the Beall Center for Art and Technology, University of 
California Irvine, December 2009 - April 2010. Regrettably Performative Ecologies did not function 
as designed during the exhibition. 



[42] See http://www.lozano-hemmer.com/displaced emperors. php 



[43] Enactivism, situated and distributed cognition are three major variants of post cognitivist cog- 
nitive science. 



[44] See http://www.youtube.com/watch?v=2kRuxutzluE 



[45] See http://www.no-surprises.de/the conversation 
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Tine experience, poetics, and politics oF public scripted space 
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Center for Digital Urban Living, Digital Aesthetics Research Center, Aarhus University 



The computer is moving out into physical and urban reality. Since Mark Weiser's call for a 
'computer for the 2P* century' in 1991 a migration from the screen and the desktop towards 
integrating computers and networks into our surroundings has been a part of contemporary 
computer science research; for example, in augmented reality, ubiquitous computing (ubicomp), 
and pervasive computing. A number of technological developments (such as big screens, new 
smart materials, GPS, RFID tags, and ever faster and cheaper wireless networks) have helped 
carry the research agendas out into ordinary reality. 



This article will discuss how we experience the urban space of ubicomp. It will do so by intro- 
ducing the concept of 'scripted space' in order to discuss how ubicomp is related to new de- 
velopments in public urban space. Focusing on the experience of the urban we will argue that 
scripted space is a concept that highlights the written, coded quality of ubicomp. As opposed to 
that suggested by such titles as 'The Disappearing Computer' (Streitz, Kameas and Mavromma- 
ti, 2007), we believe that embedding the computer into the environment will not render it trans- 
parent or invisible. Rather, computing will continue to be an important part of our conscious 
experience. Furthermore, we will concentrate on the present rather than speculating on future 
potentials, technologies, and uses. As argued by Genevieve Bell and Paul Dourish, ubicomp has 
arrived; the future is here, though it looks different than originally envisioned by the IT design- 
ers and technologists (Bell and Dourish, 2007). It is especially remarkable that ubicomp is pre- 
dominantly a 'messy' cultural interface and not, as envisioned, a 'seamless' tool for work. The 
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many parallel technologies, networks, gadgets, and software developments have become 
part of 'cultural scripts of many sorts' - for example, via the cultural developments surround- 
ing mobile phones, social software, and web 2.0 (Bell and Dourish, 2007: 141). 



Scott McQuire discusses whether this development has the potential to change public space 
and reverse its demise. In continuation of the work of Paul Virilio, Richard Sennett, and Jur- 
gen Habermas he argues that public space has lost to the public image consumed in private 
at home, and that the urban has lost to the suburban. McQuire quotes Philip Kasinitz, writing 
that suburban spaces 'are notably rich in private spaces and poor in public ones' (McQuire, 
2006: p. n.p.). But this might be changing: 'With the emergence of mobile devices, media 
consumption is increasingly occurring in public. It could be argued that the 'media event' is 
in the process of returning to the public domain' (McQuire, 2006: p. n.p.). 



Drawing on a workshop study of digital layers in the midsized Swedish city of Lund, the 
article will consider how this might happen. It will conclude by discussing the potential of 
scripted space as a new public domain by mapping out a matrix and suggesting writerly 
scripted spaces as a way of furthering this development. 



Scripted Space 



The concept of scripted space has been suggested by Norman M. Klein (Klein, 2004) but he 
does not elaborate on the concept in relation to ubicomp. Etymologically, the term 'script' re- 
fers to something written. It is mostly understood in a literary sense as either a 'manuscript', 
meaning the handwritten original, or a set of instructions and dialogue for a performance. In 
IT and software, drawing upon these connotations, scripting refers to the user's own writing 
of instructions for the computer. In an operating system, experienced users may add scripts 
that overwrite certain files, for instance, or mount net volumes. These scripts can later be 
activated automatically by clicking on an icon. In authoring software like Adobe Flash, us- 
ers may add 'action scripts' which determine dynamic behaviours of objects in an animated 
drawing (e.g., in a game where the score is increased whenever a ball hits a hole). In the 
written set of instructions, scripts have a semiotic attribute. The signs, however, are rarely 
read but are experienced as a performative attribute. The meaning of scripts is established in 
the performance of the instruction and not in the coded, incomprehensible instruction itself. 



Our understanding of scripted space is based on the fact that ubicomp literally means add- 



fibreculturejournal.org FCJ-133 111 



The Scripted Spaces of Urban Ubiquitous Computing 



ing scripts to the space surrounding us by programming smart things, architecture, infra- 
structure, and PDAs, and by connecting the scripts of these devices through wireless net- 
works. Ubicomp adds simultaneously a semiotic and a performative attribute to the urban 
space. Recognising ubicomp as a scripted space highlights how digital scripts are added to 
the coupling of space and signs already carried out through urban planning, architecture, ad- 
vertising, street signs, and so forth. As such, the scripted space of ubicomp does not result 
in an entirely new configuration of urban space, but is seen as continual development of the 
urban experience, which gets intensified by the adding of scripted environments and ob- 
jects. The scripted space is in other words related to earlier developments of the spectacular 
modern city (McQuire, 2006; Fold, 2004) and denotes an urban experience scripted through 
architecture, visual media, signs, and semiotics (figure 1). 




Figure 1: Scripted space. 



Besides being a continuation of the ways urban space has been planned, scripted space also 
has a more contemporary dimension which is linked to the computer. Scripted space has a 
non-visual, coded, encrypted side to it and suggests that there are computed transactions 
and control structures behind the facade, surface, or interface of the city. 
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This aspect has also been described by Bruno Latour and Emilie Hermant. They describe and 
photograph the development of Paris from 'la ville lumiere' to 'la ville invisible' - from the 
illuminated, spectacular city that we still visit and cherish to an invisible city with an infor- 
mational layer folded into its spectacular surface (Latour, Hermant and Shanon, 1998). The 
invisible city is a city where computers and information workers are constantly working to 
describe and script the city; Latour and Hermant document processes where the elements 
of the city (cafes, monuments, streets, etc.) are constantly under surveillance and correction, 
described and prescribed by networks of computers and information workers. What becomes 
clear from their detailed description of Paris (published before the current surge of ubicomp) 
is that the city is constantly being produced (re-engineered and mediated) by computers in 
complex networks and that it takes considerable amounts of work and material infrastruc- 
ture as well as large numbers of highly skilled workers to maintain this. 



Subsequently, the scripted space as the info-layer of the city is a complex and networked 
software structure where algorithmic scripts organise and control the city and our experience 
of it. As expressed by Nigel Thrift, the computer is 'extending its fugitive presence through 
object frames as diverse as cables, formulae, wireless signals, screens, software, artificial 
fibres and so on' (Thrift, 2004: 584). 



Following Scott McQuire, it is this mediation and scripting that holds the potential return 
of an urban public space. When the 'media event' returns to the urban public domain via 
ubicomp one should not merely consider it as disappearing into the city, invisibly and trans- 
parently embedded in the experience of the urban. Rather, the coded, scripted, or essen- 
tially written quality of ubicomp should be highlighted. Of course the scripts of ubicomp are 
hidden behind screens, devices, and interactions. In this way, they are not as accessible or 
readable as the texts in books or on signs. They are (as scripts or code in general) first and 
foremost experienced in their execution. The scripts of ubicomp add new layers to the al- 
ready multilayered urban scripted space - which is already more or less impossible to read 
in its entirety, but is nevertheless experienced through urban daily life. But how is the urban 
scripted space experienced? And how does it relate to a public domain? 



The Experience oF Scripted Spaces 



Obviously, a scripted space is experienced in many different ways, but an important as- 
pect is the feeling that something is going on behind the facade and that there are powers 
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controlling and structuring what is seen and experienced. The immediate experience of the 
urban is disturbed by the feeling that there is something unreadable, but still scripted, pro- 
gramming the space. 



In the novel The Crying of Lot 49, Thomas Pynchon describes the narrator's panoramic view 
of a southern Californian sprawling suburb in ways that illustrate this experience quite well. 
The narrator watches a dull suburb, where 'nothing was happening,' and she thinks of 'the 
time she'd opened a transistor radio to replace a battery and seen her first printed circuit. 
The ordered swirl of houses and streets, from this high angle, sprang at her now with the 
same unexpected, astonishing clarity as the circuit card had.' Since the novel was first pub- 
lished in 1966, Pynchon uses a radio circuit board, rather than a computer, as a metaphor for 
the hidden information structure. Elsewhere in the novel, though, computers are remarkably 
present, and the novel revolves around hidden information structures, networks and almost 
paranoid investigations of these. The urban space is hardly identifiable, but simultaneously it 
seems to communicate a hidden plot - a concealed information infrastructure. The space is 
not remarkable and cannot be read in spite of its 'intent to communicate' and 'hieroglyphic 
sense of concealed meaning': 'There'd seemed no limit to what the printed circuit could have 
told her (if she had tried to find out); so in her first minute of San Narciso, a revelation also 
trembled just past the threshold of her understanding' (Pynchon, 1979: 14-15). 



This 'hieroglyphic sense of concealed meaning' is an essential part of the experience of the 
scripted space, and the novel describes a very early example of an informational urbanity 
which is experienced not as readable signs, but as 'signs of signs,' as hieroglyphs that the 
urban dweller knows are readable for those with the right code but that cannot be accessed 
by outsiders. One feels that there is something going on beyond the surface, but it is difficult 
to see or read the scripts. Rather than becoming 'natural', the scripted space maintains a 
highly mediated character, even though the urban dweller does not necessarily see through 
it. A similar view is found in Norman Klein's description of the experience of scripted space in 
places such as Las Vegas, Disneyland, and the entertainment architecture of The Jon Jerde 
Partnership (e.g., malls like San Diego's Norton Plaza and urban areas like the Universal City 
Walk in Los Angeles). Spaces where 'each square foot must pay off and that bring on an 
effect of 'happy imprisonment' and 'ergonomic control' (Klein, 2004: 332). Again, the urban 
dweller feels the control and the imprisonment - she knows that she is being manipulated 
but still enters freely and enjoys the user-friendly ergonomics. 



The urban dweller's experience of concealed meaning, happy imprisonment or ergonomic 
control is comparable to other experiences of scripted spaces. Computer games seem to 
offer an important contemporary exploration of this experience of a scripted space. The 
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player can interact with scripts in different ways. On a functional level, a script can be acti- 
vated logging in to a computer game, setting control menus, etc. However, the player's most 
dominant experience is of course the aesthetic experience. On an aesthetic level the player 
activates scripts to play with them. As suggested by game theorist Ted Friedman the 'way 
computer games teach structures of thought - the way they reorganise perception - is by 
getting you to internalise the logic of the program' (Friedman, 1999: p. n.p.). In this internali- 
sation the computer's scripts simultaneously work in two ways. At once, they are they are 
an ally helping the player in controlling the environment and an obstacle to be overcome, an 
opponent. The player must collaborate with the computer in order to find a way to overcome 
it. A game experience on an aesthetic level is essentially about playing with and against the 
scripts as long as it remains a compelling activity. 



A third option is however also possible in the computer game. Friedman's description of an 
internalisation of the computer's scripts has a distinct political level. Winning a game can 
only happen by accepting the terms of the game. In Friedman's case, the game Civilization 
II (1996), the player can only win by assuming the role of a colonising nation state; art and 
religion serve a purely functional role: to keep the people pacified and so forth. The game 
thus addresses controversial historical and ideological assumptions. To Friedman, the player 
must submit to this political level but experience from other games suggests different strate- 
gies. Working as a designer for Maxis on their game Sim Copter (released the same year as 
Civilization II), the artist and founding member of The Yes Men, Jacques Servin (a.k.a. Andy 
Bichlbaum) illegally incorporated an easter egg into the game. Triggered by a cheat code or 
on certain dates, Servin's secret script will flush the game world with men in trunks kissing 
both each other and the pilots in the game. Servin's easter egg indicates a political response 
to the ideology of the scripted space. Here, the scripted space is subject to debate and will 
be reconfigured, used in unintentional ways or even hacked. 



How do these functional, aesthetic and political levels and experiences of the urban scripted 
space manifest themselves? We took part in a workshop exploring such issues in the mid- 
sized Swedish city of Lund. [1] 
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Scripted Spaces in Lund 



The forefront of scripted spaces might be more or less 'virtual' spaces, such as computer 
game spaces, which are heavily commercialised and 'themed', or it might be post-urban 
spaces, such as Las Vegas and Los Angeles, or high-tech spaces, such as Singapore or 
Tokyo. But as demonstrated by Latour and Hermant, this mediation of space also spreads 
beyond the technological 'sci-fi' spaces to affect the experience and quality of space in more 
ordinary cities (Latour, Hernant and Shannon, 1998). Ubicomp is increasingly becoming part 
of an everyday urban life - also outside the techno-sci-fi capitals, at least in the richer parts 
of the world, where the population is readily equipped with ubicomp technology and net- 
works. Lund is an example of this. 



In general, Lund consists of many layered traces of history, culture, commerce, personal and 
social narratives. We see ways of using and mis-using the urban space, and we see signs 
of transactions and interactions whose full meaning eludes us. We see signs of class and 
ethnic struggles, signs and sign systems that make people feel welcome or alienated, safe 
or threatened. Of course, our interpretation of and reaction to these signs depend on who 
we are and what we want - for example, whether we are residents or non-residents, our 
purpose, and our financial and cultural capital. Hence, a native to Lund who knows her way 
around will read the city scripts differently from a tourist. She will be able to read and in- 
terpret more of the scripts and normally also be less conscious of her readings and perhaps 
unconsciously overlook the stuff she is not able to interpret. However, this changes neither 
the basic hieroglyphic nor the everyday-like quality of the scripted space. 



As in the computer game, we found behaviour that indicated the importance of staying con- 
nected to the scripted space, scriptory control, and the potential for renegotiating the scripts 
themselves. Focusing on the digital layers of Lund, we saw three different interrelated ways 
of using public space. First, IT is used to create what we will label a log-in space, often de- 
signed to keep unauthorised access out. The digital layers are primarily visible as gateways 
under surveillance so as to restrict access. Only those able to locate the access points, who 
also know the passwords and have the keys to log in, can access the gated areas and hidden 
scripts. We find many traces of surveillance, encrypted and secured information structures, 
and keypads for logging in. Sometimes we find traces of information structures that appear 
enigmatic to us as unauthorised foreigners but that are still clearly communicating 'a hiero- 
glyphic sense of concealed meaning', such as signs indicating hidden infrastructures, security 
alarms, and surveillance cameras. We find many physical structures guiding and welcoming 
us, combined with electronic layers sorting and disciplining the clientele - for example, the 
signs in alluring shop entrances indicating the credit cards accepted, in this way welcoming 
those with the right cards and excluding those without money (figure 2). Another example 
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of a log-in space could be the list of available wireless networks on a laptop or PDA, which 
often require log-in passwords and authentication (figure 3). 




Figure 2: Credit cards in shop entrance. 




Figure 3: Wireless network log-in. 
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Second, IT is used to create what we will call a mobile personal computing space. We ob- 
served many laptops, media players, mobile phones, and other devices allowing people to 
carry their private data, soundscape, media- and communication-devices with them in public 
space (figure 4). Accessing, configuring, and inhabiting public space through personal inter- 
faces thus compete with, for instance, newspaper reading, which previously defined public 
life in a cafe as a private behaviour in public space. Also, text messages setting up social 
encounters can be considered to play an important role in the way both non-residents and 
residents make their way around urban space. When we add to this websites advertised 
on stickers or posters, virtual maps, and other web 2.0 services, it becomes clear that the 
mobile personal computing space is not only about the isolated individual (figure 5). It is the 
individual's interface to the public space. However, as implied by, for instance, Apple's suc- 
cessful range of 'iProducts' (iBook, iPod, iPhone), this interface is still privileging the individu- 
al and delivers personalised views and configurations of urban, social space. 




Figure 4: Mobile personal computing space: 
a lap top in a cafe. 



Third, IT is used for a hypertextual connectivity and linking of local nodes (people, places) 
to other nodes and the network. It is important to be well-connected, to be on-line, and to 
be accessible. In this fashion, most of the urban dwellers we passed in Lund probably had 
mobile phones in their pockets, as well as credit cards and other keys allowing them access 
to the scripted space. Cafes and shops have websites and email addresses, and some even 
provide wireless internet access, thereby offering the possibility to connect and get on-line 
(figure 6). Many urban dwellers probably also have their own networks of social links, favour- 
ite sites, and perhaps even exchange tags and preferences. In this way, an urban space and 
its dwellers are connected to each other, and to other spaces. Compared with earlier ways of 
congregating and inhabiting urban space, this social formation is thus often strangely invis- 
ible to outsiders in physical urban space. 
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Figure 5: Sticker on a pole advertising 
a website. 




Figure 6: Kiosk advertising internet access with an @. 
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The Scripted Space as the New Public Domain 



The advent of digitally scripted objects in an urban setting can be seen as a movement of 
the public domain from the private home back into the urban setting. Log-in space, mobile 
personal computing space, and hypertextual connectivity each suggests a different approach 
to the scripted space as a new public space. Mapping out the observations from Lund, we 
find a matrix of different potential approaches to the scripted space (table 1). 



SCRIPTED SPACE 


Home 


Urban 


Individual interface 


Desktops & internet 


iPhone, laptop, PDAs, etc. 
(the digital urban cafe) 


Public interface 


Weblogs, WIKIs, etc. (web 
2.0) 


Urban screens, media 
facades with public 
interaction, e.g. integrated 
with laptops, PDAs, etc. 
(the digital urban agora) 



Table 1: A matrix of different potential approaches to the scripted space. 



In a home (or an office) setting, individual access to a scripted space takes place via the 
desktop and the internet - whose historical predecessor could be either the reading or the 
inscription of the personal notebook, manuscripts, letters, and the like. Today, the movement 
of this activity into an urban setting takes place via PDAs, laptops, phones, media players, 
and so on, all of which are made for individual use in an urban setting - creating a personal 
reading and inscription of space. Using an MP3-player, a laptop in a cafe, or hypertextual 
connectivity and linking via an iPhone; getting a personal schedule, information about a 
future meeting point in town, the opening hours of a store; receiving messages, and partici- 
pating in an urban game - all of this contributes to creating an individualised, personal space 
within the urban setting. Though this kind of public behaviour could be seen as purely func- 
tional, it should also be seen as an individualisation of the urban setting where public space 
becomes a playing field for private activities - an 'individual game space'. A historical prede- 
cessor to this kind of public participation is the cafe, where city dwellers read newspapers, 
are situated behind glass windows to see and be seen, or plan their city activities through 
immediate access to city information (through city magazines such as TimeOut, gallery and 
movie lists, etc.). 



A public variant of access to the scripted space is nevertheless also evident. Quite often, 
the data accessed is not merely controlled and static but dynamic and open. In other words. 
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hypertextual connectivity and linking is not just about accessing information but also about 
actively contributing information - as labelled in the web 2.0 paradigm (O'Reilly, 2005). 
Interfacing to the scripted space is not merely individual but also collective and public. A 
historical predecessor to these cultural forms could be a participatory newspaper culture, the 
writing of chronicles, letters to the editor, and so forth. This reflects an idea of public space 
as not just a setting for individual performative activities but also a space for debate and 
knowledge dissemination - a political level of scripted space equivalent to the idea of the 
agora. 



What is striking about the observations from Lund is the emphasis on using digital technolo- 
gies for personalised access and interfaces to the urban space and the public domain. Most 
often, digital technology in urban space is predominantly exclusive. This is of course the case 
with the creation of log-in space, but also to some extent with the mobile personal com- 
puting space. However, the hypertextual connectivity and linking seem to suggest a desire 
to connect to the social. Surprisingly, today, we usually see participation in weblogs, WIKI- 
editing, and so on as a participatory culture that does not involve the urban setting. Though 
WIKIs and weblogs may be accessed and read from an urban setting, there has been little 
effort to actively promote the urban setting as a context for active participation. Surely, ex- 
amples can be found - such as user-generated content of urban screens and media facades, 
and video and textual tagging of the cityscape (e.g., as in WIKI maps) - but in the context of 
an average urban setting, they are in no way dominant. This of course signifies an area open 
to new forms of digital civic participation and will probably develop into new ways of creat- 
ing social relations and potentially new ways of developing cultural interfaces in scripted 
spaces. 



Conclusion: Writerly Scripted Space 



The digital urban and scripted space is at once a functional, aesthetic and political space. 
It manifests itself not as a grand spectacle but most often as a space where one can log-in 
(or be left out), a space for mobile personal media consumption and local/global connectiv- 
ity. Beyond being able to operate the urban scripts for functional use (logging in, configuring 
network settings, activating links, etc.) the ability to play and emancipate the digital scripts 
is crucial for the experience of the scripted space. The digital, scripted layer of the urban 
can be accessed to serve as the point of departure for individual experiences of the city as a 
personal, public playground (a borderless cafe) or, less frequently, the basis of social encoun- 
ters, communication, and debate (a borderless agora). This activity that defines participation 
in the urban public domain depends on the ability to access and internalise the scripts. It is, 
however, not the only way to participate in the public, scripted space of the city. 
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The city dweller may implement, experience, and use urban scripts, but what happens when 
she is not satisfied with the foundation of participation in urban public life, the scripts them- 
selves? This experience is a call for action where the city dwellers take control of the urban 
scripts and become 'writerly' by creating, hacking, and rewriting the city. 



'Writerly' (or 'le scriptible') is the term Roland Barthes used to characterise the particular 
way of reading associated with the open texts of modern literature. Writerly texts address 
the reader as an active producer of meaning rather than merely a passive consumer recon- 
structing the text's 'proper' meaning through a way of reading predetermined by the author 
- as in the classical characterisation of 'readerly' texts (Barthes, 1970: 10). In certain inter- 
pretations, hypertext itself is equivalent to this liberation of the reader (e.g. Landow, 1992). 
It seems, however, more accurate to evoke a digital writerly, open text as a textual corpus 
where the reader is enabled to not just experience the effect of the script but also to af- 
fect the script (by e.g. programming). Following Barthes, the German media theorist Florian 
Cramer thus advocates a 'writerly computing' where the user is transformed from a consum- 
er whose use is staged by the software interface to a producer who actively takes control of 
the scripts and has read and write access (Cramer, 2003: 100). 



Open, writerly texts/computing is the basis of all experimental digital artistic practices. This 
is where the computer is not merely seen as a medium for messages but as open to interpre- 
tation, imagination, and further production by the user. Though often not a regular sight in 
average cities, such practices that critically explore the new public domain of scripted spaces 
are numerous and include the inversion of city surveillance ('sousveillance') or other ways 
of rewriting or 'hacking' the city (Greenfield and Shepard, 2007; Fuller and Hague, 2008). 
Turning users from consumers to producers of scripts may challenge existing hierarchies of 
meanings, roles, and experiences that differentiate acceptable from unacceptable practices 
in urban scripted space. This will also mean a return of an urban public domain and a new 
paradigm of computing remarkably different from the dominant ideas of the disappearing 
computer. 
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Notes 



[1] The workshop was part of the ACM conference NordiCHI 2008, a conference on Hu- 
man-Computer Interaction, held on 19 October 2008. ( http://www.nordichi2008.orq/index. 
php?option = com content&task=view&id = 66&ltemid = 90 ). The workshop, attended by IT 
researchers, was organised by the Digital Urban Living Research Center, Martin Brynskov, 
Peter Dalsgaard, Christian Dindler, Jonas Fritsch, Kim Halsskov, and Ole Sejer Iversen. All 
participants went to the centre of Lund and were asked to look for and take photos of three 
different layers in the urban space (social, digital and physical) and organise these with ref- 
erence to three different dimensions (information, services, aesthetics) on a printed map of 
Lund. The participants were asked to focus on a transitional, mainly pedestrian space with 
shops, hotels, cafes and bars. 
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Introduction 



Presently there are a number of gameplay definitions, ranging from Sid Meier's famous asser- 
tion of 'interesting choices' to Richard Rouse's concept in which the vibrant iterations of user in- 
put and machine output is the decisive factor (Rouse, 2005). Further, 'gameplay' seems to drift 
indefinably between psychological categories, as in the emphasis on the concept of 'flow', and 
more formal conceptualisations often derived from computer science. However, the discussion 
of how the realm of pervasive games necessitates both playing and gaming, play-mode and 
game-mode, seems to oblige a slightly different gameplay terminology. One level of gameplay 
is the actualisation of a specific stratification of rules, strategies, and interactions as well as the 
realisation of a certain amalgamation of commands, plans, and paths. A second level, which is 
equally or even more important since it explains the inherent fascination of playing games, is 
the interconnection of playing in an open environment and focusing on presence and gaming 
within a fixed, discrete framework (the 'board') and paying strict attention to progression. 



The purpose of this article is to inspect the theoretical consequences of moving from a tradi- 
tional, ludological concept of computer games to an extended ludology of pervasive games. In 
the process of unfolding these consequences our general understanding of play and games — 
these straightforward and yet highly convoluted terms — will hopefully also be sharpened. Such 
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an assessment should rivet reflections that take the heightened emphasis on physical space 
as well as the contingency of socio-cultural activities in pervasive games seriously. It should 
further critically investigate the notion of the virtual. This is done in the third section where 
I employ Gilles Deleuze's Bergson-inspired claim, that the virtual is indeed more 'real' than 
any present actualisation, as a backdrop against the hypothesis that, rather than invoking 
a common sense dichotomy between a tangible reality and the informational bits and bytes 
of the computer, we should instead focus on the discontinuous relation between virtual play 
and actualised gaming as the pivotal modus operandi of pervasive games. The article can 
be read as a continuation of the work on game theory laid out in Walther (2003, 2005, 2006, 
2007a, 2007b, 2007c). Consequently, the noticeable lack of empirical references in this article 
is not the result of a normative disapproval of the many and very interesting experiments 
with pervasive technology that flourish these days, but on the contrary must be regarded as 
a rather rigid attempt to enlighten the deep axioms and often ambivalent key conceptions 
underneath the world of digital games. 



Rules and Gameplay 



In line with economic game theory we can define games as complex, rule-based interaction 
systems consisting of these three key mechanisms: absolute rules, contingent strategies, and 
possible interaction patterns. Game rules are absolute in the sense that while the players 
may question the rationality of the rules at hand, they are nevertheless obliged to obey, to 
'play by the rules'. Rules are therefore absolute commands (Neumann & Morgenstern, 1953) 
and unquestionable imperatives. They transcend semantic issues, cultural signification, mor- 
al agendas, etc. This does not, incidentally, preclude the fact that game rules are discussed 
in a cultural or ethical milieu. 



In contrast to rules, strategies are contingent, nonabsolute entities since they count as the 
more or less detailed plans for the execution of turns, choices, and actions in the game. 
Other strategies than the ones actually carried out could have been outlined and performed. 
Both in the shape of short-term tactics and as long-term schemes, strategies are contingent. 
In economic game theory, a strategy is an overall plan for how to act in the assembly of dif- 
ferent states that the game may be in (Juul, 2006). Game theory studies the affiliations of 
the rules and the strategic behaviour in competitive situations (Smith, 2006). Finally, inter- 
action patterns are the moves and choices, which become part of the game being played, 
thus interfering with the restrictions and options of the game. As the implementation of 
game strategies tend to cluster in selected regions of the possibility space of the game (in 
approximation of what is known as the 'dominant strategy' in game theory) forming a path 
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through the game space, [1] we may even insinuate that the interaction patterns, taken as a 
whole, are the game itself — especially if we view it from the perspective of the player (Hol- 
land, 1998). Interaction patterns are the possible as opposed to necessary combinations or 
the emergent outcome of rules and strategies. This differentiation can be listed even more 
briefly: 



-Rules are commands 

-Strategies are plans for game executions 

-Interactions patterns define the actual path through the game and specify the to- 
pography of human-computer (or player vs. rule) dynamics 

Clearly, the interaction patterns work as 'middle ground' as they occupy a domain located 
between the machine that upholds the rules (the computer) and the human player who has 
to find and optimise the best way to accomplish the goal of the strategy. 



The notion of game play, which we shall pursue also as regards the relation between free 
play and confined gaming, involves all three levels of a game, which also explains the dif- 
ficulty in defining the concept properly. We can refer to the following definition as the on- 
tological or formal definition. The definition is ontological because it assumes at least the 
minimal and necessary (axiomatic) existence of some quasi-material, algorithmic entity: 



[Gameplay: Definition IJ Game play is the actualisation of a specific stratification of rules, 
strategies, and interactions as well as the realisation of a certain amalgamation of com- 
mands, plans, and paths. 



For a player, a successful game play means a delicate balance between knowing the rules 
and mapping one's strategy in accordance with both rules and the possible actions of oppo- 
nents. Games should be equally challenging and rewarding, hovering between boredom and 
anxiety thereby assuring a space of flow through the network of choices. For a computerised 
game system, a successful game play implies a balance between fixed rules and the control 
of player input in variable settings. 

What defines a rule? A rule, being algorithmic in its core design, consists of a simple, un- 
equivocal sentence, e.g. 'you are not allowed to use hands while the ball is on the pitch'. 
Here, a rule constitutes the possibility space of a game by clearly stating limitations (not 
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use hands) as well as opportunities (the ball is on the pitch). It is always possible to define 
a game both in negative and positive terms: rules limit actions; they determine the range of 
choices in the possibility space; they encircle the arenas to be played in; yet they also frame 
what can be done. 



At this point, I am speaking of all games, both traditional games, including sports, and com- 
puter games. Heroes of Might and Magic rests on rules stored in and processed by a com- 
puter. Chess or Monopoly, by contrast, relies on rules not accumulated in the database and 
algorithms of a computer but written down on paper and stored in the player's mind during 
the play. In a game of soccer, for example, a referee administers such rules ultimately by ref- 
erence to the FIFA Handbook. Implicit rules that are normally considered exterior to the 'real' 
rules (e.g. a time restriction in chess matches) must be engaged explicitly in digital games. 
These rules have to be programmed as well. Weather conditions or the general physics of a 
soccer game are usually taken as 'out-of-game' features in the real world. When we simulate 
a soccer game in a computer, however, the rules of soccer and the general physics (including 
random variables such as surface granularity, crowds, time of day, etc.) must be built into 
the rule algorithms and the input-output control of the computer. 



Rules specify the constitution of the playing 'deck' or, more broadly, the playing 'field'. In 
games, behavioural patterns inside this field are limited, constrained, and highly codified 
(Huizinga, 1994; Caillois, 2001; Walther, 2003). Rules are guidelines that direct, restrict, and 
channel behaviour in a formalised, closed environment so that artificial and clear conditions 
inside the 'magic circle' of play are created (Salen & Zimmermann, 2004). The outside of this 
circle, reality or nonplay, is essentially irrelevant to game play. Confronted with unambigu- 
ous rules, strategies (or tactics) might entail best practice solutions variable to the given rule 
constraints. Hereafter, interaction patterns map the various player interventions and can 
hence be viewed as a texture of moves and choices overlain on top of the possibility space 
of the game. Furthermore, interaction patterns can refer to the social and competitive in- 
termingling of players during the fulfilment of the game. In that respect, the patterns corre- 
spond to the outcome of absolute rules and social dynamics. 



The formal organisation of games can be regarded as a parameter space. In this space, the 
current state of the game counts as a point and ultimately a dimension in the parameter 
space. A played game has therefore n possible state dimensions. In Tic-Tac-Toe, for instance, 
the nine squares constitute the parameter space of the game and thus the possibility do- 
main for the arrangement of the board pieces. The rules of the game define the possible 
edges in the space connecting states, and the total number of discrete points in the parame- 
ter space represents the total number of games states. Rules define the possible game as in 
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the initial framing of the game, whereas a particular game is a path through the state space. 
This latter particularity rests, consequently, on a variability space upon which one can also 
measure the optimum rate or success probability of the system. The crucial factor is that 
there can be no variability or multiple paths through the possibility space of a game without 
the compulsory parameters of the game. Hence, the parameter space constitutes the tran- 
scendental level of the game, whereas the particular game path expresses the contingent 
realisation of the space. 



This dialectic between parameter space and actual game path (or variability space) also 
sheds some light on why games are complex; basically it is because there is an uneven rela- 
tion between the unchanging set of rules and the actual and changing realisation of a partic- 
ular game. This asymmetrical tie between rules and realisation (or rules and strategies) can 
be termed game emergence. Most often it is impossible to predetermine the actual moves 
and outcome of a game only by knowing the set of rules. Also, most games are games of 
imperfect information (Nash 1997). At the outset, the rules of chess are simple, and yet the 
wealth of distinct chess playing tactics is quite enormous. A child can memorize chess rules, 
but to master all grand openings in the actual game is probably a lifetime achievement. 



Play-mode and Game-mode 



It is however a characteristic feature of many types of framed differentiations, or 'fiction' 
(in the widest possible sense, and notwithstanding if this feature counts as an impossible 
Utopia), that they wish to expand the gaming space, physically and mentally, often by recon- 
figuring the social landscape of worlds into a dense grid of game objects, game goals, and 
game worlds, thus obscuring the demarcations between the real and the virtual. Very often, 
stories, movies, games, and other narratives play with these demarcations. 



What, then, is most important? Is it the game itself, or is it rather the social and geographi- 
cal infrastructure that supports it? Who (or what) has the upper hand? Is it the relational 
complexities of the characters or other personified 'avatars of story' (Ryan, 2006) or perhaps 
the fluent vectors of the game world? In the following we will explore this tension between 
the telic game orientation and the presence of a world surrounding the former by drawing 
upon the recent paradigm of pervasive computing (Walther, 2007b; Walther, 2007c). [2] 



There is a fine line between being there, somewhere, and being there with a purpose. The 
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mundane space that a human subject inhabits is not by nature geometrical; rather, it is 
structured in accordance with matter-of-fact actions. In such a spatial environment, the vari- 
ous orientations are related to directions (practical vectors), places, ranges of space, and 
things, in contrast to dimensions, points, lines, and absolute objects. The space for action 
is a praxis-architecture or a phenomenological space that is not defined by length, height, 
and width, but rather by territory, proximity, and distance (Nielsen, 1996). A personal space 
zeroes in on the required equipment and relations to institute meaning, whereas a geomet- 
rical space is continuous and unbounded. 



Territorial exploration, whether in the real, physical world or in the flowing realm of one's 
fantasy, involves the incessant modification of intentions. It is an advanced procedure of 
trial-and-error set in a socio-semantic circumstance. You go right. Not interesting. You move 
to the left. Wait, here's something. You rush straight forward, and the result is immediate 
and loads of action. Such movement requires both cognitive mapping and a basic percep- 
tion of metric coordinates. However, the elusive co-existence of being present and intention- 
ally moving around for a reason is also known as rules. It subsumes at least three important 
characteristics of structured, goal-oriented activity: momentum + direction (vectorization) 
+ a valorized and quantifiable endpoint or outcome. Mapping a place through adventurous 
discovery in order to figure out the story underneath the space, and possibly inventing new 
ones in the same process, is all about playing. Learning to move and advance in a space 
filled with discrete norms of orientation, i.e. a parametrical space, meaning that you can do 
this but not that, and that you can go here but not there, is the art of gaming. 



Thus, there are two firmly interwoven modes of game epistemology: there is play-space and 
there is game-space. Accordingly, there is play-mode and game-mode. Together they form 
the much hyped and commonly misunderstood term 'game-play'. We call those games that 
mix up the tangibility of everyday spaces with the closed information spaces found in digital 
computers pervasive games. Such games may be the next generation in computer games. 
Make people move around. Don't tie them in front of the screen. Moreover, these games 
are particularly captivating because they deliberately place the relation between rules and 
world voyaging, gaming and playing, the parameters of games and the variables of play, at 
the nucleus of the very rule system itself. In other words: you learn how to master the rules 
of the game by playing them out in the real world. Pervasive gaming is game-play out in the 
open. 



In the play mode, one does not want to fall back into reality (although there is always the 
risk of doing so). In the game mode, it is usually a matter of climbing upwards to the next 
level and not losing sight of structure. Play is about presence, while the game is about 
progression (Walther, 2003). Play-space could be a city, and game-space could be the rules 
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and informational network dictating what can and cannot be done during game-play. Or, to 
rephrase this in abstract terms, play-space could be a fictitious world, with its binding rules, 
and game-space the rules and missions within this world: the teleology of the protagonist, the 
endpoint of his trajectory. 



Play is also about uncertainty, and herein lies the irreducible element in play which, according 
to Roger Caillois, makes it inaccessible to mathematics (Caillois, 2001: 173). Complete trans- 
parency derived from calculation and perfect strategy means the disappearance of player 
interest together with the pleasurable uncertainty of the outcome. 



Look at people playing. One notices that there is always the inherent but beguiling hazard of 
being 'caught' in reality. Nothing is more distressing for play than the aggressive intermission 
of reality which at all times jeopardizes play as play or simply threatens to terminate the privi- 
leges of play. Then it's back to normal life — which may be, incidentally, a giant gamespace 
in its own right, as McKenzie Work suggests (Work, 2007; cf. also Galloway, 2006). This is, of 
course, a structural feature of all play and of all game-play. This is true of chess and soccer. 
It is also apparent in Doom and Myst. Interruption and termination must be avoided at all 
costs — in the continuous pursuit of having fun — but, since they are inescapable, they must be 
built into the very 'being' or ontology of playing games. 



Now, consider pervasive gaming, game-play out in the open. As a player I rush down a street 
in order to amass my next item to be uploaded via my PDA so that my game-buddy at home 
can keep track of my doings and goal-seeking so far. It's 4 pm, there is heavy traffic, and I am 
momentarily barred from reaching the corner with the alacrity I wished for. 



We witness a growth in the design of game systems that use ubiquitous computing tech- 
niques to propel forward player experiences that connect objects within the real world with 
objects of the computational world. SuperFly, by the Swedish game company It's Alive Mobile 
is a good example. The player's aim is to become a virtual celebrity. The projects Can You See 
Me Now? and Uncle Roy All Around You, both created by the UK performance group Blast 
Theory, use hand-held digital devices, GPS location tracking, and online agent technology in 
an attempt to use location and mobility as game features from within the real world. While 
one player stays at home and moves a virtual character around a representation of a real city, 
other players speed around the real streets, trying to hunt down the virtual quarry. 
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In chess, there are no strident interruptions between two or more discrete fields. I move my 
queen independently of physics, be it weather, traffic jams, or the occasionally bad habits of 
my fellow citizens. In a game of soccer, you block your opponent, and he tries to tackle you. 
However, a nice set of training principles that look for ways to avoid the physicality of block- 
ing is always an option. That is what the refinements of dribble are all about. In pervasive 
game-play, mixing play-space and game-space, 'real' problems, as the ones described above 
in my own thought experiment, remain real problems. If not, the aesthetics of producing 
eloquent game mechanics turns into a matter of ethic. I do not, in the quest of fulfilling the 
game's teleology, knock down the old — real — lady on the sidewalk only because she is re- 
fraining me from targeting the 'pac man' further down the road a little bit faster. 



Therefore, we must be careful in judging the fun factor of game-play. It is not only the city, 
the social and geo-graphically expanded context, in itself, that is the locus of enjoyment in 
pervasive game-play. Yes, I can go explore, and yes, I meet people, and yes, the site of navi- 
gation has become much wider than a trivial board. Nevertheless, the bouncy guarantee of 
space might indeed become the constraints of the game. Serious gamers do not want to 
waste their time looking for 'interesting' places to explore. They much rather want to under- 
stand the structure so as to move forward revealing new game areas or climb upwards in the 
hierarchy of levels. 



As we shall see later on, this veiled and all-important prerequisite of playing games, including 
the pervasive ones, is contained within the notion of 'the virtual'. Being a conditional causa 
sui of the actualised game, and the gameplay that unfolds in the being-present, the virtual 
explains in its pure form the dual and much overlooked nature of gameplay: the virtual is 
the 'past' of the 'now' of gaming since we must always silently remember the enjoyable and 
playful offspring of a game, and, at the same time, the virtual drives the articulated though 
momentarily non-fulfilled target point of gameplay that is the 'future'. 



Why? Because play is centred in a discovery of open spaces that invite observation through 
the duration of temporality. Gradually, one learns how to pilot inside play, and since the com- 
pletion of more and more successful tasks takes time, it corresponds to the distinctive forms 
that keep differentiating the play system into finer grades of subsystems. One inhabits spaces 
like these via certain as-if structures; one assumes a role and lives out characters whether in 
the form of other players or agents that one can adapt as a player. The gamut of play equal- 
ises a measurement of its geometry — how big is the playing field, and where are its borders? 
And these lengths and widths become in turn the source of gaming's internalisation of both 
geometrical space and discrete progression. 
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In contrast, play seems to focus on investigations of semantics, since the task is, not only to 
measure its space, but furthermore to elaborate upon its modes of interpretation and means 
for re-interpretation. Not only do we explore a world while playing, its potential meaning and 
the stories we can invent in that respect also drive us. Play spaces tend to expand, either in 
structural complexity or in physical extent. This expansion is further reflected in the praxis of 
play, for instance when players argue over the exact thresholds of a play domain. Another 
feature that distinguishes playing from gaming is the notion of presence, as I pointed out 
earlier. Obviously, the sensation of presence is tightly interwoven with phenomenological 
concepts like 'immersion' and 'flow. Play commands presence. We have to be there — not only 
be there, but also be there. This is the double meaning of Heidegger's dasein: dasein (be- 
ing there) and dasein (being there). We go with the flow; or, rather, while swallowed by the 
presence of playing we are in the flow, as Mihaly Csikszentmihalyi claims (Csikszentmihalyi, 
1990). A game's success is intimately tied to the organisation of space and time. Gamers 
need to trust this organisation. Since a game hinges on a certain finite structure in order to 
promote infinite realisations of it — the correlation of rules and tactics — the very articulat- 
ing of presence so important for play must already be presupposed in a game. One already 
knows in a game that the mission is to keep on gaming, which really means, in my vocabu- 
lary, to keep on playing, that is, to prolong the sensation of presence. The energy can then 
instead be directed towards elucidation of the game's structure: 'How do I get to the next 
level?' and not 'Why do I play?' This keeping on, knowingly or unknowingly, is the virtuality of 
games. 



Although one should indisputably respect the ethical boundaries of pervasive games that 
transport game-play out in the open — just as one should bear in mind that the metaphysics 
of fictional worlds often goes beyond the natural laws and moral confines of everyday life; 
one does not want to hang on too long for the old lady to cross the street. While waiting, the 
question above might turn up thus threatening to disintegrate the exquisitely balanced halves 
of gaming (to progress) and playing (to be present). 



Thus, we can put forward the second definition of gameplay. In continuance of the first one, 
we can refer to the following definition as the epistemological or player-oriented definition: 



[Gameplay; Definition 2J Gameplay is that kind of player activity that intentionally involves 
the asymmetrical relation between world exploration and level progression. 
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What do the novel features of pervasive games, including play-mode and game-mode, 
mean with respect to rules, as we have defined them above? 



-Rules are commands. However, since pervasive games often take place in a physi- 
cal context that is characterised by socio-cultural contingency, rather than the dis- 
crete controlling of parametrical progression, rules should therefore be seen as deep 
commands within a variability space that, simultaneously, allow for the more or less 
intentional modification of rules. 

-Strategies are plans for game executions. However, while strategies in strictly 
confined 'boards' with clearly defined edges may be easier to control (and hence 
to realise), strategic actions in large pervasive games building on the complexities 
of both real and computational space tend to evolve into even more complex sub- 
strategies and perhaps unforeseen interaction patterns. 

-Interaction patterns define the actual path through the game and specify the 
topography of human-computer (or player vs. rule) dynamics. True, but while the to- 
pography of traditional computer games rests on a topological level structure which 
enables, for instance, a virtual teletransportation from one 'dimension' to another, 
the topography or path of large pervasive games — typical of urban gaming — is 
rather based on the geometrical constraints that are bound to the actual physicality 
and linked with the socio-cultural contingency of the gameworld. 

In the following we shall qualify the notions of play-mode and game-mode further by relat- 
ing them to Deleuze's concept of the virtual. 



Virtuality 



A prevailing notion of 'virtuality', which was especially dominant in the cyber theories and 
virtual reality oriented writings of the nineties, sees it as a kind of spatial and epistemo- 
logical liberation from Cartesian geometry (Ostwald, 1997; Lunenfeld, 2000). The kind of 
space that Descartes had in mind, in his Meditations on First Philosophy, is organised in 
accordance with Euclidian mathematics and categorises spatial elements in a general and 
absolute sense (Descartes, 1996). In Euclidian space there is no resistance; it is an ethereal 
morphology containing absolute, non-contingent relations. A typical cyber theory goes on 
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to claim that the spatial form of the computer (and its spatial representation) more willingly 
obeys the laws of differential topology, which describes spatial singularities or catastrophes. 
Cartesian space is analogue with its emphasis on measurable planes, geometrical continuity, 
and friction-less dimensions, whereas the topological space of Henri Poincare is digital and 
discontinuous. The effect of the latter is the displacement of spatial orientations (such as 
up/down and inside/outside) and the novel possibility of constructing latitudes with infinite 
dimensions that threaten to adjourn the space as object. 



Moreover, the virtual has generally been considered as the apex of a media and artistic 
evolution. Thus, an important distinction between static images in which the perspective is 
tied to the observer, and virtual reality implemented, interactive environments, is the user's 
liberated command over spatially distributed point of views. Ever since the Italian Renais- 
sance and the theories and works of art by Alberti, Leonardo, and Botticelli, the painted arts 
have tried to master a linear vista so as to create depth, visual consistency, matte relations, 
and smooth continuity. But the idea of artistic creativity in the Renaissance nevertheless 
rests on the artist and the way he manipulates the experiences and senses of an audience. 
The celebrated trompe I'oeil is a technical culmination of this conceptual and visual striving 
for perfection that can be regarded as a pre-form of augmented reality: a frame within the 
frame or an illusion within the illusion itself. Later, in photography, the point of view is still 
linked to physical setting of a camera in concrete space-time. Motion picture and television 
finally release the perspective into moveable and oscillating events in time. Not only do im- 
ages move around in front of the camera lens — the camera is itself mobile in relation to the 
actual viewpoint. Yet, this dynamic mode of representing and presenting space-time has a 
certain limitation in perspective because the photographer or director completely determines 
the POV. It is only with the advent of virtual reality 'that the user can have substantial visual 
control of the scene' (Bolter, 1996: 113). 



Gilles Deleuze's definition of virtuality and the virtual that he inherits more from Aristotle and 
Spinoza than from Plato and Descartes, and which he elaborates particularly in Bergsonism, 
Difference and Repetition and The Logic of Sense, is remarkably different from the above. 
Interestingly, he seems to arrive at a classification of the virtual — including an assumption 
of metaphysical grounding, to which I will return — not because of the concept in itself, but 
rather as a response to a problem with ontological implications posed by structuralism. How 
can the underlying structures or systemic constructions exist if they do not belong to either 
the mind of a subject or the material world of objects? Deleuze's answer is that these struc- 
tures, which act as conditional guidelines for present actualisations, represent something 
that is, ontologically speaking, 'in-between' subject and object, and this is exactly virtuality. 
How is it possible, Deleuze asks, for something to be a condition of being, to be a catalyst of 
an actualised articulation or presence, while not being discernible or measurable, or being 
located in one particular person's mind, or otherwise embedded in a material world? The 
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difficulty is to establish the exact ontology or existence of such relational, determining struc- 
tures, i.e. to insist that this ontology is eo ipso the most comprehensive form of reality and, 
at the same time, a detached and left behind reality. Part of the answer lies in Deleuze's de- 
liberate Kantian design. Similar to Kant's notion of space and time as unifying forms imposed 
by the subject that do not, in themselves, exist in space and time, Deleuze holds that pure 
difference is non-spatio-temporal — reality without actualisation, ideality without abstraction. 



The reason why 'the virtual' commonly appears obsessively contrary to reality is because we 
fail to acknowledge virtuality as the 'real' condition of actuality, and also because we one- 
sidedly prioritise the actual as the being-present over the virtual as pure Being. The virtual is 
neither non-existence nor ontic occurrence; rather, it is a real system of differential relations 
that creates actual spaces, times, and sensations. Deleuze writes in Bergsonism: 



We have [...J confused Being with being-present. Nevertheless, the present is not; 
rather, it is pure becoming, always outside itself. It is not; but it acts. Its proper ele- 
ment is not being but the active or useful. The past, on the other hand, has ceased to 
act or be useful. But it has not ceased to be. Useless and inactive, impassive, it IS, in 
the full sense of the word: it is identical with being in itself (Deleuze, 1988: 55). 

Could it be that 'play' belongs to the virtual and 'game' to the actualisation of being? And 
what does that mean in relation to the philosophy of pervasive gaming that seems to be 
more attentive towards explorative play than the traditional emphasis on level progressive 
gaming? Let us pursue this thought in the following. 



At least in traditional computer games, play-mode works as the virtual or relational condition 
for the present activity and the ontological materialisation we call 'the game'. This condition 
is more than the underpinning algorithms and digital codes but must be considered the ab- 
stract, or even ideal, base of games as such. There is a condition, a raison d'etre upon which 
the materialisation of each game depends and that condition is play. One could argue that 
play is an ontological praxis; but it is certainly also an ontological condition. Continuously, 
new games arise and old ones cease to be — not just in an inventive but also in a pragmatic 
sense. Contrary to this constant activity in game-time, the inexorable pastime of playing 
games, 'play' has the strange quality of being discontinuous in time: somehow it is there all 
the time, as a completion of the idea incarnated in a specific game, and, at the same time, it 
is not there since it cannot be found in the integral whole of any particular object. The game 
is the discrete being or entity that seems to discriminate the conditional past (play) as ab- 
stract and fictional. However, as Deleuze states in chapter four of Difference and Repetition, 
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'Ideas and the Synthesis of Difference': 



The virtual is opposed not to the real but to the actual. The virtual is fully real in so 
far as it is virtual [...] 'Real without being actual, ideal without being abstract'; and 
symbolic without being fictional. Indeed, the virtual must be defined as strictly a part 
of the object — as though the object had one part of itself in the virtual into which it 
plunged as though into an objective dimension (Deleuze, 1994: 208f). 

Virtuality is not the unreal in a stark contrast to the actual existence of either ideal subjectiv- 
ity or material objectivity but, rather, the determined structure or 'embryonic' element that 
completes the actualisation more than makes it whole. Following from this we can say that 
a game is a discrete, unconnected and actualised entity that belongs to one and the same 
relational structure or system — the latter being 'play' or the Bergsonian pure past. While 
the game is experienced in time as a discontinuous, ontological emanation, play accounts 
for the process in which we, unknowingly, move from the virtual domain of ideal relations 
and singularities that characterise a system to uniquely actualised entities. Whereas games 
constantly move about and reorganise themselves into discrete actualities with epistemic 
qualities, the 'purity' of play withstands this perpetually passing-by through a strong sense 
of simultaneity. Play, then, completely determines a game but is only a part of the object. 



There is of course the danger that 'past Play' becomes a secret transcendental teleology 
of 'present Game', or that the virtual-as-Substance turns into a Spinozian metaphysic. In 
temporal terms, the actual (which we call game-mode) corresponds to chronos, the pure 
present, whereas the aion, or the virtual (which we call play-mode), is the pure past and the 
condition of chronos. A certain impassivity therefore clings to virtuality as an abstract onto- 
logical memory that insists in all actuality and yet can never be said to contribute anything 
to the concrete instances of our chronological time. We would then fall prey to the Spinozian 
idea that everything that exists is a modification of the one substance. 



Does this imply that games cease to exist the moment they occur and that, by contrast, play 
is composed of the real substance that somehow endures despite the constant fragility of 
relational organisation? No, insofar as play is the 'past' of gaming's 'presence', this does not 
mean that play contributes nothing to actuality and disappears as a tragic but true Being. 
Rather, the two levels, substance and actualisation, playing and gaming, the ontology of the 
virtual and the epistemology of the present, exist in a continuous state of flux, a nomadic 
unrest that perpetually reproduce the relational encounters from moment to moment and 
from one game state to another. This is especially true in the later writings of Deleuze — the 
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one who turns to chaos and complexity theory — where the distinction between the virtual 
and the actual progressively disappears, and we instead get interrelating processes where 
events reproduce certain patterned relations or organisations. 



Taking off from Deleuze we can pursue the proposed link between the virtual past and the 
being-present of gameplay. I suggest reading the Bergsonian framework, not as a compre- 
hensive metaphysics of time, but as a conceptual scheme of causal conditioning. The follow- 
ing figure illustrates the double realm of virtuality. 



VIRTUALITY 

past 



Fig. 



pUy 



AmiAT.TSATION 
present 

» 4 

gamcplay 



MRTIIAI.ITV 
future 



game 



First, the virtual accounts for the underlying past of the actualised game — resulting in pres- 
ent gameplay — as it separates itself from the other of play, i.e. reality or nonplay. This is 
how we described the virtual in the above. However, there is a second and just as important 
characteristic of virtuality in that it also occupies the teleological domain of games. Here 
we find the level oriented progression and goal of gaming and, in a way, this very modality 
of the virtual signifies the irrefutable 'arrow' of play and games. Traditional videogames as 
well as novel pervasive games always possess this telic attribute — heading, with or without 
detours, for the quantifiable outcome and the Utopia of winning. Furthermore, the two poles 
of virtuality, both pointing directly towards the now of gameplay, are intimately coupled with 
the transitional localities in games. Dangerously near the edge where a game exactly stops 
being fun to play, play-mode is a feeble confine because of the latent likelihood of stepping 
out of the magic circle and back into reality. Game-mode bears a similar stamp of transi- 
tional delicacy due to the all-pervading thread of blocking the uni-directional gameplay thus 
freezing the player between two levels and denying him the bliss of advance, access, and the 
potent omen of triumphing. 
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As I see it, the Deleuzian outline represents a step forward from the premature and overtly 
hyped understanding of virtuality as the ontologically 'un-real'. Rather than to focus on 
whether the virtual, in the traditional cybertheoretical sense, dictates an essentially different 
metaphysic or a whole new set of phenomenological qualities, we should enquire into the 
differentiating and imposing forms of play and games as they mutually determine the logic 
of virtually conditioned actualisations in time and space. This way, we do not need to come 
up with any new metaphysic or phenomenology. Furthermore, this approach emphasises 
the virtual as the transcendental, non-spatio-temporal, and driving force behind the praxis 
of play with nevertheless actual and real implications. The virtual is not play, but it in-forms 
its being. The virtual is not (the) game, but it shapes and sharpens its imminent horizon. 
Virtuality becomes a conceptual tool with which to describe both the playful past and the 
goal-oriented future of gameplay — thus being a kind of synonym for the what if and as if 
of the homo ludens. In fact, this is an anti-Deleuzian claim; rather than the call for a 'Body 
without Organs' that thrives in playspace like a madman amidst the open-ended, smoothed 
immanence of nomadic anarchy (Deleuze, 1987: 149ff.), the underlying thesis here is one of 
an inevitable return to a striated, grid-fixed game universe. Ultimately, 'flight lines' becomes 
telle lines; not necessarily unmovable guidelines, but guidelines nonetheless. 



Gameplay is the actualisation of rules, strategies, and interaction patters as well as a non- 
equlibrious poise between explorative play and level-oriented gaming. Pervasive gaming, as 
we have seen, revitalizes exploration as the blood and bones of naturalised gameplay; and 
yet it never completely abandons the telos of the discrete, parametrical, and competitive. 
The virtual domain, including the past of play and the future of games, is hardly ever ques- 
tioned as such in gameplay; the conditioning forces always ensure the unsaid and proper 
framework within which play and games can take place. This way, the virtuality of games is 
perhaps not just the logico-formalistic precondition of gameplay as it hovers between the 
mimicry of play and the teleological, rational desire of progression, but a violently imposing 
'discrete ideology', as Slavoj Zizek would say (Zizek, 2008). [3] The constant reterritoriali- 
sation of the virtual that happens in the course of playing games is thus a play within the 
virtual much more than it is a play with the virtual. And, as it happens, Zizek hates Kung Fu 
Panda: 'If you ask me for really dangerous ideological films, for ideology at its purest, I'd say 
Kung Fu Panda. I saw it five times because my son likes it. The movie is extremely cynical in 
that you know they make fun of all this ideology, of Buddhism and these things, but the mes- 
sage is even though we know it is not true and we make fun, you have to believe in it. It's 
this split of you know it's not true but just make like you believe in it' (New York Entertain- 
ment, 2008). 
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Notes 



[1] It is a commonplace of screen-based, contemporary computer games to equate this 
dominant strategy with graphical trajectories so as to ensure a dominant path in the oth- 
erwise seemingly 'free' world of the game. See Espen Aarseth's analysis of Half-Life 2 for a 
comprehensive documentation of this aspect (Aarseth, 2005). 



[2] I should stress that when I am referring here, and in the remainder of the article, to 'per- 
vasive gaming' I exclude persistent games (Everquest, World of Warcraft) as well as Alter- 
nate Reality Games and MMORPG's, on the one side, and mobile games (existing computer 
games reshaped for cell phones and other handheld devices) on the other. Thus, pervasive 
gaming represents a technological paradigm that relies on adaptronics, wearable, mobile, 
or embedded software and hardware in order to facilitate a 'natural' environment for game- 
play that ensures the explicitness of computational procedures in a post-screen setting. As 
a result, pervasive games frequently use GPS, various types of wifi (and Bluetooth), and 
signal triangulation techniques. In true pervasive gaming the physical environment must be 
prepared, technologically, for mobile, location-oriented gameplay. Consequently, there is a 
fine line between the natural environment that facilitates the played game and the natural 
environment that is responsive to gameplay. 



[3] I am not claiming that Zizek should be against games on the above grounds — in fact, 
he isn't (cf. http://scrawledinwax.com/2008/09/15/wax-scrawls-zizek-on-ideology-and-vid- 
eo-games-crashing-parties-and-sms-art/). I am merely suggesting that a certain Adorno- 
Foucault inspired critique of the underlying instrumentalism of competition and masculinity 
clearly predominant in the culture of play and games will find it relatively easy to pinpoint 
the 'unknown knowns' of our object. Sadly, any thus far proposed counter strategy that 
would work against the ideological powerhouse of gaming — attempting to subvert the 
violent logic of games; desiring to compose non-teleological gameplay; invoking 'tragedy' 
rather than the usual potent endings of games — is precisely against the point, that is, out- 
side the magic circle. 
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Introduction 



In 'The Coming Age of Calm Technology', Mark Weiser and John Seely Brown are clear in their 
assertions. What really 'matters' about technology is not technology in itself but rather its 
capacity to continuously recreate our relationship with the world at large (Brown and Weiser, 
1996). Even though they promote such an idea under the banner of 'calm technology', what is 
central to their thesis is the mutational capacities brought into the world by the spillage of com- 
putation out from its customary boxes. What their work tends to occlude is that in setting the 
sinking of technology almost imperceptibly, but deeply into the 'everyday' as a target for ubiq- 
uitous computing, other possibilities are masked, for instance those of the greater hackability or 
interrogability of such technologies. Our contention is that making ubicomp seamless (MacColl 
et al, 2002) tends to obfuscate the potential of computation in reworking computational sub- 
jects, and thus societies, modes of life, and inter-relations with the dynamics of thought and the 
composition of experience and understanding. 



Even though we might recognize this inherent demand in the bifurcation of 'calm' technology 
into the new feral logics, ubicomp's reliance on the development of artificial intelligence - 
through techniques of identification, naming, tracking, sorting, monitoring and responding - has 
been precipitated, though if not always inspired by, a research agenda inherited from military 
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programs inaugurated during World War II onwards (Heims, 1980; Agre 1997). This same 
'military' heritage continues to define much of the research agendas of significant academic 
and industrial institutions, directly influencing their epistemological orientations. Doing 'calm' 
requires then an overemphasis on the machine, keeping the deployed human component 
stable and unexcited and focusing essentially on the development of artificial intelligence 
frameworks that in turn calmly interpret and act on the calmed user. Such an approach can 
be seen to operate in opposition to a deeper understanding of human-computer symbiosis, 
one that may even step out of this dyad to engage with a wider understanding of systems 
and ecologies. What is crucial to recognize here is that this agenda, infatuated with an 
ontological positivism simple enough to teach a machine, establishes distinct conceptions 
of ubiquity that ramify into computational technology, human action and cognition and the 
environments in which these occur. As it stands, many of the problems experienced by ubiq- 
uitous computing as a progressive research program can be found in the reliance on such 
an ontology and its apparently stable concatenations of misidentification and mismatched 
response, delimitating the potential of new logics into simplified models of calculation. 



Despite its partial 'military' heritage, ubiquity is effectively bifurcating from this calculative 
approach, challenging much of the practice of computer programming and other layers of 
the ordering of computers. This is now diversified by: networks, embedded systems, new 
graphical user interfaces, the World Wide Web and wireless devices, networks of sensor- 
actuators, and the multi-form variations introduced to computing by its deep embedding in 
the social technical, biological, aesthetic and political dimensions of life. This renewed con- 
text has, particularly within the field of computer science, shifted from a view of 'computa- 
tion as calculation' to one of 'computation as interaction' (Stein, 1999; Wegner, 1996; Mur- 
taugh, 2008). It has deviated from the core of its first-order programming philosophy, from 
computation as 'number crunching' to an object based and distributed approach (Wegner, 
1996; Gouldin and Wegner, 2003). The growth of computation into a new feral state has also 
affected the ways in which user relations with the new artifactual ecologies can be under- 
stood, extending previous research frames. Here, the emphasis on the ideal of 'computation 
as calculation' has not only affected the field of computer science, it has also influenced the 
field of cognitive science (Dupuy 2000) and those concerned with computation as metaphor, 
instrument, field or infrastructure. 



Once again, ubiquitous computing and its propensity for distribution as part of environments, 
opening new space for variable kinds of users and contexts, has shifted the idea of cognition 
as analogous to the workings of computational devices to an idea of cognition as situated, 
embedded and distributed. This idea has much in parallel with the field that Heinz von Foer- 
ster and others named 'second-order cybernetics' (Poerkson and von Foerster, 2002). Indeed, 
amongst other contexts, this field has a lasting influence, or even undergoes a revival in 
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some of today's laboratories of computational design, one example being in the develop- 
ment of robotics (Bishop and Nasuto, 2005). One of the main goals of second-order cyber- 
netics was the attempt to study complex systems, including humans and machines, under 
a new light. It attended in particular to the way in which they formed patterns of reflexivity, 
gained a recursive sense of self and of the wider processes in which that self co-composed. 
Crucially, they also emphasized the study of systems that are analytically indecomposable, 
such as memory, or which grow as part of a 'conversation' (Pask, 1976), or other form of 
structural coupling (Maturana and Varela, 1998), making the neat distinction between sub- 
ject and object, observer and the observed, simply a neat, but disabling perceptual gimmick. 



One of the founders of the cybernetic movement, Norbert Wiener, posed the need to in- 
terlink the new worlds of 'automata' with distinct social interests and concerns, where the 
'decentralization of authority should accompany the decentralization of computation'. This 
project would soon be obfuscated by scientists such as John von Neumann, whose vision of 
their use tended to relegate computational power exclusively to the interests of the 'military 
and industrial establishments'. This limitation inaugurated at the same time cybernetics' 
disengagement from domains - unfortunately - characterized as 'humanist', finally closing 
the computer into itself (Eglash 2000). As a consequence, the inherited 'military' agenda 
has contributed to the divide between cultural approaches to technology and the means for 
a technical imaginary and those that characterize themselves as 'scientific'. Today, an ex- 
pansion of the scope of interdisciplinary approaches within the distinct domains that study 
human-computer interaction and computing cultures more widely is moving towards a 
reversal of this 'trend'. Here, ubiquitous computing has played a decisive role. Its propensity 
for distribution throughout the environment has opened space for a new range of contexts, 
users and artifacts, raising new complex questions that extend beyond the premise of per- 
fecting artificial intelligence. What once was the concern of an engineering driven discipline 
is now delivered to a wider field of intelligences and skills beyond any discipline. 



Interaction, the Extension oF Computation beyond Calculation 



Both the fact and ideation of ubiquitous computing and its characteristic embeddedness 
has driven the need for an exploration of new human-computer relations, shifting, in one 
description, the research motto from 'proactive computing' to 'proactive people' (Rogers et 
al., 2006). The shift arises not only from a new conception of human/machine symbiosis - of 
the sort originally theorized and supported by Licklider (1960), towards mutual proactivity 
- but also from a challenge from within the field of computer science itself. Essentially, this 
shift has been accompanied by a transformation of the understanding of what computation 
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is and how it might be done, dislodging a model in which computation is seen as a series of 
fixed functions that are outlined to achieve a certain specified goal. Computation as some- 
thing 'centralized, sequential and result-oriented' and primarily focused on the execution of 
calculation, moves tentatively or eagerly, but perhaps inexorably, towards an understanding 
of 'computation as interaction' (Stein, 1999: 1). Here, dynamic relations become key. A com- 
putation does not simply equal the achievement of a result, measured by an ever increasing 
metric of acceleration towards that result, but tends towards a collaboration with the user 
or other elements in a wider ecology, understood through a connective multiplication of the 
capacities of each entity in the computational composition. 



Where the 'calculative' approach has persisted - in the disavowal of computational artifacts 
beyond their formal description as variations upon calculus - this has tended to make dif- 
ficult or even prevent the recognition of any external influence, or of the multi-layeredness of 
computational situations. Concerning the embeddedness of computational gadgets through- 
out the environment, it has particularly influenced the ways in which certain dogmatic 
computational frames have pushed forward a 'result-oriented' rather than 'process-oriented' 
programming styles (Stein, 1999). Such imperatives have often dissuaded engaged profes- 
sionals from elaborating any form of 'computer criticism' that could provide a richer frame 
for understanding computational artifacts and their contexts (Papert, 1990). At the core of 
such reluctance sits the bulk of our conceptions concerning the practice of programming, 
ideas that are embedded in strong cultural formations of thought and rationality. An exam- 
ple that throws light on both epistemological styles and their consequences for computation, 
the Logo programming environment for children - a pioneering project of its kind - was built 
upon the recognition that many computational environments, for the purpose of education 
still preserve canonical dogmas, such as the fixation on outcome or result achieved by pre- 
determined means. On the contrary, the Logo environment has shown that many children 
favor a style that contradicts the formal canonical one identified with planning and reason- 
ing, opting for a more 'intuitive' mode; both are conceptually valid (Papert, 1990). The ex- 
ample of Logo is problematic in the sense of its limited domain of actual use, but the way 
in which it opened the possibility for different programming idioms to come to fruition and 
to be recognized as being idiomatic, with different perspectival weightings and affordances, 
is significant. Today, different programming methodologies, scripting environments, frame- 
works, and the increasing sense of coding and computation as part of popular cultures and 
interesting subcultures open computation to a multiplication of conceptual and idiomatic 
registers (Mackenzie, 2006). 



In fact, such conceptions of programming based on ratiocination have a long history in 
the development of the computational field itself. These find their philosophical legacy in 
a historic battle between rationalists and empiricists. Descartes' figuration of reasoning as 
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the base of existence was followed by Kant's denial of the contingency of true knowledge, 
and in turn by those contemporaries who, in their most useful formulations, find the rational 
folded into and entangled with the aleatory, intuitive and experiential in ways that defy the 
capacities of expectation of either model. Still, much of modern Western science has been 
influenced by rationalist mathematical modes of thought, these even more intensified by the 
successes of technological development, the faith on the dependency of such rationality, and 
a corresponding ability to recognize a world which is increasingly reformatted into rational- 
ity's own image. 



The history of Western thought, however, is also a history of successive collapses of total 
formalizations (Kline, 1980). Kurt Godel's 'incompleteness theorem' that exposed the fissipa- 
rous faultiness of rationalism when turned upon itself is only an example, but a crucial one 
in the history of computing. The thesis opened the way for a new epistemological agenda 
where the idea of incompleteness would prevail, a decisive element in bringing the paradigm 
of interaction to centre stage for the development of computing. Alan Turing famously took 
up Godel's 'incompleteness theorem' and in 1936 published a paper 'On Computable Num- 
bers with an application to the Entscheidungsproblem', showing that 'mathematics could not 
be completely modeled by computers' (Goldin and Wegner, 2003: 1). However, whilst ac- 
knowledging its foundation in the tracing of limits to computability, the Turing machine was 
adopted by computer scientists at large, providing a seductive model with which to address 
problems. Computer science took the transformation of 'inputs into outputs' as a defining 
characteristic of computation, a transformative process already known to the mathematical 
field and best described by formal algorithmic operations (Goldin and Wegner, 2003: 1). 



Approaching the 1990's, computer science faced new challenges. With a new logic of ubiqui- 
ty, from the World Wide Web to wireless devices, phones and other more cranky assemblag- 
es, new questions appeared. Logical-axiomatic transformations (as understood through the 
'inputs into outputs' of the Turing machine) as the sole answer to computational problems 
(those problems to which computing was addressed and distinct therefore from 'computable' 
ones) would no longer suffice. By that time, many already felt the need to extend computa- 
tion into new paradigms, as the novelty of machinic formations evolving out into diverse 
ecological conjunctions allowed the exploration of, for instance, physical effects rather than 
solely logical ones. Today, one result of this is the development of 'interactive-identity ma- 
chines', simple transducers characterized by non-algorithmic behavior that use the comput- 
ing power available in the surrounding environment, substituting mathematical reasoning 
with empirical development (Wegner 1996). Here, the new 'interactive' paradigm appeared 
as most attractive, not only since it best explicated possible relations with 'real' environ- 
ments that could not be completely predetermined by a list of configured inputs, but also 
because it provided the best extension for the already achieved capabilities of the Turing ma- 
chine. The twist being that the 'incompleteness' now supplied by the interacting environment 
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is a feature that is not under the machine's control, or not integral to its axiomatic nature, 
but generated out of its points of (intensive and extensive) conjunction with the world. 



The new interactive ecologies could not rely on formal computable algorithms. These en- 
gaged in a constant switching back and forth, between the figure and ground of computa- 
tion in the machine, and in the potential for computability within the environment. There was 
an increasing demand for developing new dynamical models (Stein, 1997) to new interactive 
algorithms (Mackenzie, 2009), extending to 'computation beyond calculation'. Most impor- 
tant, 'interactive computing' allowed the necessary resistance to the canonical style, reject- 
ing computer science's legacy, finally leaving space for a new epistemological orientation. 
Of course, it is not that suddenly a computer becomes something more than a technically 
describable object, rather, computers were turned inside out. Their processes and affiliations 
routed themselves through non-computable processes producing new moire patterns of reso- 
nance, interaction, and interference. 



The Interactive Paradigm, a Cybernetic Revival 



We can also trace the origins of interactive approaches to the 'Macy conferences' held be- 
tween 1946 and 1953, in New York, when a group of researchers met with the goal of dis- 
cussing 'Circular Causal and Feedback Mechanisms in Biological and Social Systems'. Here, 
the term cybernetics was taken on, following the title of mathematician Norbert Wiener's 
(1948) book on control and communication (Eglash, 2000). Two of the interdisciplinary re- 
searchers present, mathematician Norbert Wiener and neurophysiologist Warren McCulloch 
inaugurated what later became characterized as the 'first-order' cybernetic movement. The 
research motto: the idea that the dynamic entities, which maintain certain kinds of consist- 
ency over time, such as the human nervous system, can be characterized by internal feed- 
back processes that maintain constant stability. 



Part of the research agenda of cybernetics was an emphasis on inter-disciplinary applicabil- 
ity through the creation of meta-categorical concepts, such as feedback, that crossed both 
living and non-living entities. Cybernetic approaches, based on powerful degrees of abstrac- 
tion, were later applied to the development of new kinds of artificial intelligence, now framed 
by a connectionist approach that emphasized structures of simpler but interconnected neural 
units (Bishop and Nasuto, 2005) instead of intelligence deriving from one main central pro- 
cessor. This 'bottom up approach' was applied to both machinic computation and human 
cognition. Aspects of this approach are paralleled by what is often recalled as 'second-order 
cybernetics', epitomized in, but not limited to the work of physicist and philosopher Heinz von 
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Foerster and participants in the Biological Computing Lab at the University of Illinois. Sec- 
ond-order cybernetics is characterized by finding ways in which feedback loops travel out- 
side of the boundaries of an entity, affecting its behavior in reflexive and non-determinable 
ways. Here, von Foerster introduces the idea of 'subjective dynamic construction' (Poerksen 
and von Foerster, 2002: 23) into the complex web of feedback loops within a given system. 
This abstract framework relocates the position of the observer or agent of a given system 
into its dynamics. If for first-order cybernetics the agent was decoupled, with second-order 
cybernetics this would no longer be possible. Such a position re-describes, with a different 
scale of reference, the relevance of situating computational artifacts as 'external scaffold- 
ings' (Griffiths and Stotz, 2000), where agent and environment are no longer fully separated 
but mutually articulating. 



In more recent times, there has been a growing interest in using von Foerster's ideas in the 
field of robotics. Key factors have been the importance of environmental situatedness and 
non-representational embodiment as the driving forces in the development of intelligent 
behavior. Additionally, 'second-order cybernetics' opened the door for what later would be 
called a 'dynamic systems theory of cognition' and the 'enactive theory of perception' (Bish- 
op and Nasuto, 2005). For both, cognition is a highly interactive process where agent and 
environment have important and active roles. Building on, but also questioning, earlier theo- 
ries of environmental perception, such as those of James J. Gibson (1977), which are well- 
known in design, a dynamic systems theory of cognition recognizes the importance not only 
of the relation between an agent and its environment, but also the 'state-space' that relates 
such entities and the permutations of actions that compose them. 'Enactive perception' rec- 
ognizes the role of embodiment in the process of cognition. The agent only perceives to the 
extent its perceptual system enables it to do so. For both approaches, cognitive processes 
are determined by agent-environment coupling dynamics (Bishop and Nasuto, 2005). Reality 
is no longer an established edifice but rather subject - but not reducible - to an individual's 
interpretation. In the case of research performed within ubiquitous computing and its de- 
pendence on artifacts that track, monitor and respond to a set of human activities, the idea 
of agent-based reality puts in question, once again, the idea of a predetermined computa- 
tional entity. 



The transformation of computation from 'calculation' to 'interaction' also has a direct rela- 
tion to the development of some of the main ideas developed by von Foerster, Gordon Pask, 
Gregory Bateson and others associated with second-order cybernetics. Turning intelligence 
inside-out, this model opposed the idea of simple internal representation, proposing instead 
one of external and distributed pragmatics working with affordances. Such conceptions have 
in turn influenced computation itself, recomposing the idea of programming as a bottom 
up process, a move that returns to influence ideas about intelligence. Instead of something 
being sited in the 'head' (Noe, 2004), it has an emergent quality depending on high levels of 
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interaction. Again, there are strong levels of filiation. The 'bottom up model' was first devel- 
oped by cyberneticians such as Wiener, based on 'corresponding' connectionist approaches 
to cognition. Second-order cybernetics challenged this view, by locating the agent in relation 
to the same system (Bishop and Nasuto, 2005). This further promotes a 'constructivist episte- 
mology' where the observer, perspectively and enactively, constructs her own experience and 
cognition becomes a continuous dynamic process. 



At this point, it needs to be acknowledged that a new conceptual approach to the material- 
ity of the world is also devised. First, this provides a means of recognizing the position of 
both agent and environment in a complex dynamic ecology, where the extensions or limits of 
either are not clear-cut, or are rendered effectively meaningless. Second, it provides a means 
of abstraction in which behaviors and patterns that cross categorically distinct entities can 
be recognized. Finally, it was a form of thinking, based strongly in practice, which opened up 
hybrid insights into forms of life, providing a new methodological approach when researching 
the 'social' and the 'natural' (Eglash, 2000), or let's rather say, with apprehension of their in- 
adequacy as terms, the 'natural' and the 'artifactual'. Concerning the design of computation- 
al artifacts, such an agenda presupposes the relative importance of both user and artifact, 
now appearing in the light of more tricky, multivalent relations. It also presupposes that the 
user is as much responsible, or at least implicated - in multiple, non-predictable ways - in 
the construction of their own artifactual ecology. Such a constructive process calls for an ap- 
proach that places computation in a new light, reemphasizing the importance of a research 
agenda that marks the expansion of computation through distinct layers of interactivity. This 
entails a transformation, not only for computer science itself, but also in the conception of 
new computational artifacts and environments. 



The implication of the user in the expanded computational environment is however not a 
story of simple unfolding opportunity. The history of cybernetics is both a history of systems 
of control and of a mode of understanding and synthesizing systems of recursive self-organ- 
ization. A society of extended interaction is certainly being put into place, the computer is 
spread out into the world, computation is enfolding itself into all layers of life: but in many 
cases as systems of control. Writing from an island whose society puts more faith in surveil- 
lance cameras than in the citizens they watch we have no reason to welcome the spread, 
distributed, or open arms of control. Instead, we might usefully look towards coupling an 
understanding of interaction with a critical and inventive politics. As an applied science, 
cybernetics became in part, like early strains of HCI, a means of integrating humans with 
machine systems (Hables Grey 1995) or those of the poetry of management (Beer 1975). In 
the nineteen-eighties, one of the resulting figures, that of the cyborg, became subject of wry 
critical celebration, most famously in the work of Donna Haraway (1991). The current wave 
of re-engagement with cybernetics tends to emphasize not so much its application in sys- 
tems of control (Holmes 2007) but its empirical work amongst epistemic questions with the 
production of experimental devices (Haque 2007). 
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From Cybernetics to Interactive Designs 



Such epistemic questions have instigated the need to understand interactive technologies 
in their actual use, blurring the modern divide between the 'social, symbolic and subjective 
from the material, real, objective and factual' (Latour, 2008: 6) avoiding any absolute bifur- 
cation of interaction between users and artifacts and the relations between them. This has 
been the true contribution of the field of participatory design, one that opened the field of 
computation, finally setting up the interactive paradigm to reach out into the 'situated, inter- 
pretive and messy' aspects of human nature (Harrison et al., 2007). Forms of participatory 
design have long questioned modernity's formalism and rationality as raised on the research 
edifices of WWII and beyond. It has questioned computer science's formal approach and the 
rational structuring of cognition by incorporating user and environment, cultural, economical 
and political contexts. In this sense, participatory design makes possible, if it does not al- 
ways deliver, a 'third and hybrid space' (Muller, 2003: 2) between users and artifacts, inaugu- 
rating a dialogical plane for action. As in 'distributed cognition' (Hutchins 1999, 2000), both 
user and artifact come to play determining roles in the mix of effects and interactions, with 
the explicit recognition of the processes of thought in relation to design. For designers at 
large this is of substantial relevance, since new forms of computation are bound up with the 
encouragement of new forms of sustainability, development and experimentation. Such con- 
siderations are especially potent when we consider the contexts in which users and technol- 
ogies are embedded, along with the variable kinds of access different users have today - to 
the panoply of computational artifacts, processes and services (Crang et al., 2006). 



In this same context, it seems pertinent to attempt to consider innovation. Here, the ideal 
still ostensibly driving most technological research agendas is not so much innovation any- 
more. Instead, it is uses (Shapin, 2007) or even misuses. For this same reason, participa- 
tory design philosophy and methods have been taken on by some companies in the devel- 
opment of user-centered approaches to innovation (von Hippel, 2005). Its principles have 
been applied, in different interpretations, in a number of ways and sites ranging from the 
development of industrial artifacts to that of software. However, these are not without their 
problems. Improvements that only travel inwards to a company will soon make users rightly 
cynical. The world of marketing is not innocent of simulating 'demand' in order to generate 
it and interaction will inevitably face many kinds of shaping by interested parties and social 
and libidinal forces (Lyotard, 1993). Nevertheless, distributed innovation as it articulates the 
reflexive extension of interaction is a potent feral force that proliferates on the outskirts of 
policies and regulations, amongst distinct communities of users/designers. This same shift 
has provided a new distributed conception of innovation and knowledge, with a significant 
move to test and extend the ways in which Free Software derived development methodolo- 
gies and principles might produce effects in other domains of production and use, such as 
non-executable data. For reasons such as this, we use the term 'interactive designer' as 
much as that of 'interaction designer'. 
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As a counterattack to this potent feral force, many new approaches to the development of 
regulations and copyright policies have tended to 'imprison' the same users by relegating full 
control to corporations or owners of 'intellectual property'. These regulations, and sometimes 
the closure of designed objects, tend to retard innovation by limiting the scales and kinds of 
access to technological and cultural artifacts and processes. The rise of new computational 
technologies tends however to set this back, essentially by providing information and inno- 
vation opportunities to communities that may extend beyond 'traditional' geographical or 
economic constraints. The affordability - often at the cost of abusive forms of production, 
of computational gadgets and the ease with which people start to manipulate them, and 
thus perhaps break out of the role of simple user - is opening some space for a new era of 
technological innovators. Indeed several researchers have suggested that a democratic ap- 
proach to technological design might have the tendency towards increasing opportunities, 
or instigating new forms of economic development (von Hippel, 2005). Approaches to ubiqui- 
tous computing that develop significant modes of interaction will necessarily also meet with 
the question of the way in which data of all kinds moves about, whether it is shared, sold, 
wrapped in protective seals, or is treated as simply part of the wider ecology of materials. 



Feral Computation and Design 



It now seems pertinent to ask: what might design facilitate or enter into combination with 
in this process? Can design catalyze a process of material reshaping, further influencing 
those described above? First, we need to extend this venture and consider relations between 
designer, the artifact and systems it engages with (that now might appear in a planned or 
unplanned manner). This is perhaps even more relevant when seeking feral qualities in the 
hardware domain. Here we note the recent and widespread increase in interest in tinker- 
ing, making idiosyncratic technical objects, and the extension of this into a renewed perhaps 
quasi-popular engagement with electronics through platforms such as Arduino (see http:// 
www.arduino.cc/) and others that provide a means of recognizing and working with a quite 
palpable hunger for a more difficult, unpredictable, customizable and intelligent inter-rela- 
tionship with electronic design, often that more commonly found at the level of prototyp- 
ing. Such moves provide not only interesting options for design curricula but also for those 
engaged in design beyond formal education, as useful information and experiences move in 
ways that build self-organizing practices. This in turn establishes possibilities for a new gen- 
eration of designers, and more broadly of designing practices. By such means, as machinic 
platforms, knowledge and skills, coupled with modes of thinking that provoke ubiquitous cu- 
riosity rather than calmness, certain principles of hackability are extended to a wider range 
of users who in turn modify them. We can only envisage potential future uses when bits and 
pieces become more robust, or usefully weak enough to make interesting connections, and 
are globally widespread, opening paths for new modes of commodity distribution, hackability 
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and self-production. Here, design is involved in a more complex web of experimental devis- 
ing, moving beyond the provision of simplified solutions, engaging in the construction of a 
system that continuously reshapes the questions that were initially waiting to be answered. 



The qualities of ferality can also be found in applications such as 'Squirrel'. Designed by 
Shannon Spanhake, (of the California Institute for Telecommunications and Information 
Technology) this small device incorporates a battery, sensor chip, Bluetooth and accompany- 
ing software, 'Acorn'. The device allows users to monitor and read levels of carbon monoxide 
in the air through their cell-phones (Ramsey 2007). This extends the potential of pollution 
monitoring, once only in the hands of small public and private groups of specialized research 
enterprises. It is now opened up to users with the potential for hacking the politically en- 
closed debate. In a similar vein, Mexican artist-engineer Gilberto Esparza shows us the pos- 
sibility of designing parasitical robotic artifacts, simple life forms that feed on power gener- 
ated by human societies. The parasites 'cigd' (colgado) and 'dbit' (diablito) live suspended 
on telephone cables and re-circulate their energy while interacting with the surrounding 
environment through the emission of unusual sounds. The 'ppndr-s', a redaction of the name 
pepenadores, are small robot parasites that live amongst the accumulated remainders of 
human disposals, carrying out simple tasks such as removing, scattering and sweeping (for 
more information go to: www.parasitosurbanos.com). These artifacts live out of surpluses 
and engender a sense of the uncanny in urban situations, feral robots that couple with rick- 
ety media ecologies and the leakages and disarray of cities. Thinking of new forms of rela- 
tion between natural processes and those that are deeply synthetic in a way that can open 
them up to wider understanding, or at least more tractable difficulty, Hague Design & Re- 
search presents 'Natural Fuse', an eco-physical network that employs the capacities within 
the environment, such as plant carbon-sinking, to create a recursive system where energy 
can only be derived from electricity networks if sufficient plant matter is present in the circuit 
to absorb the carbon produced by the energy used, (for more information see: www.haque. 
co.uk ). In all these examples, hackability is taken into a new speculative dimension. Hack- 
ability itself becomes feral and no longer solely pertains to the material and infrastructural 
domain, in which it is measured by the relative technical openness of a system, but also to 
a performative one, in turn proposing new user modes for appropriating and experimenting 
with a wider sphere of interests. Finally, acquiring alternative sets of dimensions, scales and 
sizes of intervention, these new user modes call upon us to reinvent what the city is, what 
the 'proper' way of allocating energy might be and to circulate the capacity to know in new 
ways, demanding in turn new kinds of agency and knowledge circulation. 



Rather tending to contradict mainstream perspectives on the sites of technological innova- 
tion, this call has been heard from economically impoverished countries, where salvaged 
machines imported from the West, or sourced direct from fabrication in the East, have, due 
to material scarcity, been incorporated into a tradition of maintenance and redesign. By 
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transforming technologies in their own terms users come to generate a body of knowledge 
unforeseeable by the 'original' designer. The story of Morris Mbetsa, a self- taught inven- 
tor from Mombasa, seems pertinent. Mbetsa invented the 'Block & Track', an anti-theft 
device and vehicle tracking system based on mobile phone technology. The device uses a 
combination of voice, dual-tone multi-frequency (DTMF), signaling and SMS text messaging 
technology to control a vehicle's electrical system providing the user the ability to activate 
or disable the ignition in real time (AfriGadget 2008). Such ingenuity, in 'developing' coun- 
tries, is not simply a means for survival, but a crucial resource for educational, economical 
and cultural development, which shapes its own conditions of emergence. 



Interaction, Society and New Distributed Systems 



To shift to another scale, such reflections become even more relevant if we consider the 
pace at which computational artifacts have become simple everyday objects for many chil- 
dren making ubiquitous or distributed forms of computing 'everyday' experiences in ways 
that test their stability. New forms of computational literacy suggest that the relation of 
users not only towards technology, but also to knowledge production itself, will change in 
unforeseeable ways. Not disinterestedly perhaps, Microsoft is amongst those keen to argue 
that developing forms of knowledge will increasingly require more and new forms of 'com- 
putational literacy' (Microsoft Research, 2005), to which new forms of critical and inventive 
intelligence will need to arise. There is a fundamental and pressing need for the design of 
research tools that might enable or constrain new forms of experimentation and knowledge 
production. Even though, for Microsoft's vision of the future, computationally literacy will 
imply profound mathematical thinking (Microsoft Research, 2005), it seems rather prob- 
able that computational literacy will also imply a form of 'concurrent' thinking (Stein, 1999) 
that is able to couple recognition and understanding of formal processes with their other 
experiential scales and dimensions. Computation, in the light of such an interactive concep- 
tion, will of course have implications for education, challenging it with the evidence of new 
modes of experimentation. Here, materials will need to be modular and flexible, in order to 
elicit a constructive approach to both content and knowledge development. Finally, interac- 
tivity will imply novel forms of enquiry: by providing programmable and flexible tools that 
encourage ingenuity and reflexivity. 



To achieve these goals much has to be done, for example when reflecting on the ques- 
tion concerning the implementation of information technology in school curricula. Whereas 
much effort has been directed towards the distribution of computers amongst pupils and 
ensuring fast connections to the World Wide Web, little has been discovered concerning the 
actual forms in which information technology might develop and enhance useful learning 
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models (Gdrdenfors and Johansson, 2005). We once more return to 'feral' understandings 
of computation, cognition and design, and it seems that the same kinds of thinking might 
usefully be applied to education. The sphere of knowledge production, and therefore also 
partially, perhaps rather too partially, of education, is relevant to the question of interactive 
design in the expanded sense. Education's involvement in the production of knowledge prac- 
tices and of intellectual, organizational and spatial subjectivation has accompanied, if implic- 
itly, much of the debates concerning the understanding of computation (specifically artificial 
intelligence and its figuration of the position and kinds of intelligence) and the studies of 
cognition sketched earlier in this paper. Add to this the commonplace that: 'there is nothing 
more political than education' and the models we choose in the development of knowledge 
will have implications for the development of design and of societies. 



Now, if the capacities for thought are distributed between people, artifacts and the ecologies 
they are in, make and wreck, the question of learning becomes a crucial one in the imagina- 
tion of what is computable as much as what is thinkable. The consideration of education 
and knowledge in society suggests that ubiquitous intelligence already forms an uneven and 
fissiparous challenge to the simple proliferation of ubiquitous computing on all levels of life, 
and will itself demand or instigate new social and organizational approaches. The interactive 
approach, in becoming feral, articulates not only computation, but modes of social organi- 
zation, learning and alternative modes of collective innovation. Here, we recall a cybernetic 
approach to systems development where: 



... one of the key ideas the general theory embodies is the principle of recursion. This 
says that all viable systems contain viable systems and are contained within viable 
systems. Then if we have a model of any viable system, it must be recursive. That is 
to say, at whatever level of aggregation we start, then the whole model is rewritten in 
each element of the original model, and so on indefinitely.' (Beer, 1973: 4) 

This would mean that a feral approach to technological development would entail less for- 
malised approaches to society at large. It would at one scale perhaps imply the formation of 
cybernetically reflexive collectives, a 'recursive public' composed of modular, but ungridded, 
elements, where distinct modes of subjectivation, technicality and organization co-cooperate 
in the devising of their own infrastructural logics (Kelty, 2008). Finally, design and cybernet- 
ics might find a common path, design with its emphasis on 'participatory' approaches and 
cybernetics in its consideration for the development of complex systems where the system 
cannot be separated from acts of observation and involvement. 
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For designers, considering such a principle of recursion means accepting a major respon- 
sibility in the development of systems, and also the responsibility of constantly reflecting 
back to their own sphere of activity, accepting and testing the task of actually design- 
ing design, and in so doing accepting their rather more minor role. Doing so requires an 
acknowledgement that creations devised by distinct professions might actually slip out, 
escaping description by initial intentions (Fuller and Hague, 2008). This implies a deviation 
from formal and discipline-oriented professional approaches to a more multivalent view, 
encouraging design to become more than a purely sectional interest in the life of objects to 
actually consider and take part in the political, social and cultural implications of its activity 
(Wood, 2007). Such a position imagines an expanded role for design as an activity, but also, 
conversely, it suggests a recognition of the wild expansion of the scope of people, thoughts 
and things involved in the activity of design and the selection, development, deprecation 
and implementation of technological systems. 



Some Closing Remarks 



Throughout the present work we have moved rather too rapidly - but hopefully suggestively 
- between distinct fields of enquiry, from computation, cybernetics and finally design all un- 
der the rubric of an 'interactive paradigm'. We have attempted to argue that this paradigm, 
emerging from computation's enfolding into the world, has prompted conceptual develop- 
ments leading to new questions and formulations. However, central to our theme is the fact 
that none of this would have been possible without accounting for the development of new 
artifactual, technical and ubiquitous realities, where distinct artifacts have gone feral, inter- 
weaving with the subjective, social and ecological fabrics of our cities, lives and societies. 
The idea of ubiquitous computing, its characteristic embeddedness, but also partial contain- 
ment, within diverse multi-scalar settings of many different kinds, has suggested the explo- 
ration of human-computer relations in ways that can no longer be seen as simply calm, but 
also partially wild, if they do not drive us so. 



By transforming our understanding of what computation is and how it might be done; shift- 
ing from a view of computation as 'centralized, sequential and result-oriented' or as 'com- 
putation as calculation', towards an understanding of 'computation as interaction' (Stein, 
1999), we are able to alter preconceptions concerning the practice of programming and 
computing itself. This implies a shift from the views of a science that once took the transfor- 
mation of 'inputs into outputs' as essential, to one that now emphasizes a process orientat- 
ed view of computation that extends beyond prescriptive algorithmic understanding (Weg- 
ner, 2003). Due to its strong link to cognitive science, inaugurated through the behaviourist 
school during WWII, and vice-versa, such transformations have found wider implications 
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for computation and beyond. For cognitive science particularly, this has meant the prolifera- 
tion of new approaches to the study of the human mind. Starting from the premise that the 
idea of mind itself extends beyond the confinements of 'head and skull' (van Gelder, 1995; 
Chalmers and Clark 1998), the importance of the organism's environment can be recognized 
as crucial in establishing the complexities of any considered cognitive unit. The relations I es- 
tablish with my surrounding world are as much relevant in the study of my cognitive capaci- 
ties as the workings of my brain. By considering its extension beyond the single body of an 
organism, such an approach challenges the webs of rationality and formalism that can only 
be sustained when we consider the intellect as something separate from the world I am con- 
stantly immersed in. It does not mean however that such boundaries cannot themselves be 
effective when enforced or entrained by other, wider, means. All such boundaries, in them- 
selves, have meaning. 



Such conceptual workings, that transverse both computing and cognition are more readily 
understood by revisiting their filiations to a third approach: the works of second-order cyber- 
netics and its study of complex systems, including both humans and machines. This is a field 
of study and practical experiment epitomized by the introduction of the idea of 'subjective 
dynamic construction' into the complex web of feedback loops within a given system. Such 
'subjective construction' cannot be made an account of without considering the characteris- 
tic embeddedness of any organism, entity or practice. Ultimately, this has lead to the devel- 
opment of the recognition of human-computer relations as highly situated in a way that their 
cultural, subjective and political dimensions cannot be ignored. 



A proper conceptualization of the message handed down from second-order cybernetics 
might finally open new approaches to the study of human-computer interaction at large. 
Essentially it allows us to recognize that this same relation extends beyond the tight formal 
coupling once devised by behaviorist and formalist approaches to both cognition and com- 
putation. By considering the messy and informal aspects ('subjective construction') of hu- 
man computer interactions, previous Fordist models of design seem to become obsolete and 
the trajectory lead in part by the 'participatory' approach sheds new light into the design of 
alternative ecologies of computation and interaction where user, artifact and designer, play 
roles that are shiftingly determinant rather than fixed. 



If we initiated our discussion by speaking of ubiquity as key instigator of a new interactive 
paradigm, one that brought novel approaches to computing, cognition and design itself, we 
have attempted to finalize this dicussion by considering how ubiquity might embody this 
same feral quality. In this light the feral artifact is one that has been able to escape a do- 
mesticated and captive state. It is not a sedative, exuding calmness for an already under- 
stood, tracked and pre-empted subject whose needs and requests have been mapped out 
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and whose life as such corresponds to that of a finite state machine. To design for a feral 
context is to work with processes of subjectivation that are not simply described by input 
and output, no matter how far away they are from traditional sites of computation. Instead, 
such a design works with artifacts, processes, ecologies, people, computation and politics, in 
a way that is able to explore the multiple potentials of the fields of interaction, calculation, 
control and cognition. When computing escapes from being unable to recognize itself out- 
side of the formalisable, becomes promiscuous, starts finding itself recomposed in combina- 
tion with packs of other epistemic currents, it becomes feral. When design recognizes that its 
expertise is distributed amongst people, institutions, media, infrastructures and in the dis- 
sensus, joy and confusion of contemporary lives and their technological avidity and starts to 
find means of generating a rigor in the chaotic, it too starts to lose its domestication. In do- 
ing so, recursively, it promises to produce something that renders their complex artifactuality 
sensible and open - with all kinds of difficulty - to design: to think with things. 
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Background 



The rise of the ambient brings new directions in environmental criticism. Here 'the ambient' 
means a continuum of contexts where information has been embedded locally, to facilitate 
being brought in and out of focal attention. How may design for that state contribute to neces- 
sary shifts of worldview? Do new relations of embodiment, touch, tagging, and display change 
environmental awareness, no so much of global problems as of simply being in and part of the 
world? 



This word 'environmental' has been badly abused; it seems to mean everything to some people 
and nothing in particular to others. So long as whatever this word meant was considered large 
and far away, absent of technology and beyond human agency, it was difficult to imagine much 
less act upon. So long as information technologies have been considered placeless and nonma- 
terial, they have hardly been part of such imagination. 



Information has always had environmental effects, nevertheless. Abstract systems of access 
and distribution have physical, even geographic manifestations, and sometimes direct map- 
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pings. One most dramatic example of these might be America's Jeffersonian grid, a system 
of survey that encouraged commodification of land. More recent information technologies 
not only map environments but also interpret them. Data visualization techniques have 
advanced public environmental education. Then too, information infrastructures themselves 
have ecological footprints. Any of these relations of the digital and physical might suggest 
background for any of these essays in this collection. But here, let us narrow the scope to a 
question of coupling embodied information to shifts in environmental criticism. 



For as humanity now changes not only its means, but also its ends, economic valuation 
shifts dramatically, and in the process creates a greater role for design. As the upkeep of 
non-fiscal capital (e.g., natural, human, or cultural) becomes a more viable enterprise, design 
emphasis shifts from extent of material production toward more intensive, systemically inte- 
grated patterns of living. As in a traditional culture where waste was less of a possibility, the 
new environmental economy produces a sense of simple elegance that is everywhere. Here 
the wish for distraction declines, and the sense of participation increases. It is a simple truth 
of urbanism that people take care of things and places that they value aesthetically. 



Given these overarching concerns, here are four threads for consideration in this essay: 



1) Consider a turn in environmental history, which admits not only of urbanism but 
also of ambient media. 

2) Ask whether an information environmentalism is desirable, even possible, or 
whether it is mainly censorship. 

3) In the context-sensitive practices of the expanding discipline of interaction design, 
what might already be environmental, in some larger sense, at least implicitly so? 

4) Under what themes does the discipline develop a critique of these values, atti- 
tudes, and practices? How might this follow from developments in literature and the 
arts? 



1. A Turn in Environmental History 



First, a turn in the field of environmental history generally concerns itself more with artifice, 
and specifically investigates the relation of built environments and cultural values. Although 
such interest was once perhaps countercultural it now enters mainstream debate. 
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Environmental history asks how human affairs transform the living world. This has of course 
become the central question of our time. So that others might ask better how else human- 
ity might live, environmental history first describes the effects of particular ways in which 
particular peoples have already lived. While lands may indeed shape cultural dispositions, 
environmental history reverses these relations to look at how cultures and destinies have 
shaped the land. Like much else in early environmental work, it has simply tried to document 
impact. 



Such studies of setting and circumstance may seem normal to any discipline with ethno- 
graphic roots. Interaction design has been one of these from the start. Yet even quite recent- 
ly all this emphasis on place would have been considered unusual, nostalgic, and perhaps 
misguided, in just about any field. Circumstance was something to transcend. Modernity was 
the general belief that humanity could impose abstractions on the world until those became 
the world, at least in human experience, and especially through the use of technology. 



To admit of circumstantial impacts was not what most historians did anyway. As Donald 
Worster declared: 'In the old days, the discipline of history had an altogether easier task. 
Everyone knew that the only important subject was politics and the only important terrain 
was the nation-state' (Worster, 1988: 289). 



Environmental history thus belongs to a more fundamental turn toward the study of every- 
day life. This breakthrough has generally been credited to the Annales school of economic 
historians (e.g., Bloch, Lefebvre, and Braudel in France). Much of this took a voice that was 
countercultural, in which material circumstance was mainly evidence of political marginaliza- 
tion. Environmentalism inherited that voice. 



Those of us born into an age where awareness of environmental crisis is an ongoing way of 
life may have difficulty imagining that it was once uncommon to admit environmental sensi- 
bilities of any kind (Buell, 2003). It took a while for enough people to shift frame of reference 
from what were almost entire instrumental, pragmatic forms of value to admit intrinsic forms 
of value, or to understand humanity as an intrinsic part of a living world. But meanwhile it 
was being forgotten, by some quite saturated in technology, how it was just that immer- 
sion that so shocked early environmental critics. Hence the resonance of California essayist 
Henry Miller's epithet (1970): 'the air-conditioned nightmare.' 
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Considering the first decades of 'doing environmental history,' Worster emphasized how in 
the origins of this field, artifacts had been excluded. 



The social environment, the scene of humans only interactive with each other in the absence 
of nature, is therefore excluded. Likewise is the built or artifactual environment, the cluster 
of things that people have made and which can be so pervasive as to constitute a kind of 
'second nature' around them. (Worster: 293) 



This is where the field necessarily began. 



Thus another early step toward an environmental history of ambient information was to 
admit of any positive role for design. Without this, the countercultural origins of environmen- 
talism too often reduce into naive wholesale oppositions of nature (good) and artifice (bad). 
There are people who believe that any human intervention detracts from a place - even, say, 
the Golden Gate Bridge. 



More general histories of the city did not lack awareness of environment. Many a mythology 
began from the founders hallowing a chosen spot. Geography was destiny. City air, for all 
its stench, was said to make men free. Yet it was the stench that demanded a new breed of 
historians. Environmental history of the city was history of plague and pollution. Industrial 
urbanism seemed not only separate from the world, but completely in opposition to it. 



Now, much of this has been changing. For one indicator of this change, note 'Green Man- 
hattan' (Owen, 2004). Hardly an ideal place, and more like the early environmentalists' very 
idea of what to avoid, Manhattan turns out to be more green than most of America when its 
huge carbon consumption is normalized per capita. Relative to their exurban counterparts. 
New Yorkers have less stuff, perhaps more fun, and lower ecological footprints, all without 
risk of cholera. And not to get ahead of the story, but many of the benefits of mobile and 
embedded media incidentally first appear in places of highest social density. 



It is instead the exurban domains where environmental knowing seems most absent. Some- 
where in suburbia somebody has never even gone for a walk. Also it is in the rural domains 
where the worst perversions of the biosphere are perpetrated by agribusiness. To early envi- 
ronmentalists, the extent to which exurban artifice would disable common sense was not yet 
conceivable. 
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Thus the next step in environmental history was to treat the relationship of planetary crisis to 
settlement patterns, as if not all urban artifice were bad, and some ways of living healthier 
than others. Do poorly built environments undermine the very ability to think environmen- 
tally? It is ever more useful to ask how artifice can dramatize human agency and belonging 
in the world, not just provide conquest, distraction, or escape. 



Urban environmental history, once considered an oxymoron, asks how settlement patterns 
present and reflect choices about worldview. Although often beginning from the ecological 
impact of economic production, it can proceed to questions of consciousness concerning just 
what about a production or its side effects is considered necessary. It asks what has been 
valued as commons. Whatever its foundations in pristine nature, it also studies questions of 
liveability. 



This has enhanced a tendency within media, which have often been self-referential. Given 
the inherent quality of digital media to include both information about itself and links to 
other media productions, media studies increasingly investigate information as a world in 
itself. For some young people steeped in electronics, history goes back only so far as elec- 
tronic recordings. 



Histories of information, which became abundant during the web boom, have thus empha- 
sized long term change in literacy, and not the environment. Yet literacy itself can be situ- 
ated. Once it was solely in monasteries. Commercial television was first rolled out in taverns, 
to take another example (McCarthy, 2001: 5-7). Where you read a book affects how you read 
that book. The locations in media which actually get used deserve more study. This provides 
one way toward environmental criticism. 



2. InFormation Environmentalism? 



Second, a question of information pollution has long existed but now accelerates. Media fa- 
cades especially stir public debates, not only legal but also philosophical. A sense of unprec- 
edented distraction keeps growing. 



To send your first-grader on a school field trip, you might have to fill out twenty lines on a 
form, over breakfast that morning. At work your inbox is stuffed, not only with spam (which 
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can be filtered), but also with reply-to-all-recipients email from your colleagues (which can- 
not). Over lunch, there's no fashion crime in wearing a hands-free phone on one ear. To get 
the attention of a colleague who is playing his iPod loudly enough that you hear it too, you 
might have to raise your voice. After work you might have trouble finding a bar without tele- 
vision. You could carry a 'TV-B-Gone' universal remote, which scans through the IR spectrum 
in which most TV on-off signals lie, but you might be nervous about where you can get away 
with using it, and of course that would be yet one more gizmo to manage. 



Twenty years have passed since Richard Saul Wurman coined the term 'information anxi- 
ety.' What does it take, he asked, to put the verb 'inform' back into the noun 'information?' 
(Wurman, 1989: 38). Context, for one thing. Few people claim that the ubiquity of wisdom 
has been rising so rapidly as ubiquity of data. Since those twenty years are the very ones in 
which information technology exploded into everyday lives, this expression may be worth 
reexamining. How has concern for 'information anxiety' canceled or reinforced other more 
basic environmental sensibilities? The more that its designs are embedded, the less the field 
of interaction design may ignore these issues. 



Information anxiety feels like an inability to keep up, an urge to research your every move, or 
a duty to provide useless data. It comes at you over call lists, customer notices, and ubiqui- 
tous advertising. But not all information anxiety is junk. It is also your own inability to re- 
strain your grazing on news, digging into the practices of your trade, or researching exhaus- 
tive minutiae of some hobby like knitting, that you supposedly do to relax. 



In reconsidering Wurman's original thesis, the continuum of attention assumes more im- 
portance. One man's signal is another man's noise. 'What we choose to read and what we 
choose to ignore are, therefore, some of the most critical decisions that we make, yet they 
are invariably made with little thought, almost unconsciously' (Wurman, 1989: 204). The 
technical capacity to shift focus of attention has accelerated since then. Conversely, the 
social impetus to stay informed has been rendered absurd, both by the explosion of data and 
by the unforeseen capacity to search much more, and retain much less. Thus the relation- 
ship of environment and intent comes to the surface. To what does the context invite you to 
intend. How does the inscription of datascapes onto physical architectures invite intentions 
about being there instead of tuning out. How does the infoglut discount foreground mes- 
sages in favor of a more ambient 'affect.' 



'Ambient information anxiety' demands a new concern in the human sciences. Educators, 
ethnographers, sociologists, and psychologists now debate the intellectual and emotional 
impacts of a fulltime, ubiquitous, multitasked feed. Architects and urbanists need to find 
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their take on these issues as well. How does design of the physical environment mitigate (or 
worsen) information anxiety? When is it pollution? 



In Technics and Civilization, Mumford observed that 'the first mark of paleotechnic industry 
was the pollution of the air' (Mumford, 1936: 167). The neotechnic phase, which anticipated 
what would later come to be called cybernetics, shifted 'from destruction to conservation' 
(255). For example, in nineteenth century Pittsburgh, smoke and fire were discussed aplenty, 
but to construct them as pollution was to think of them as a correctable nuisance; and this 
only happened much later. In a familiar cycle within technological determinism, pollution is a 
social construct. 



Today a mother at the supermarket takes great care to scrutinize the ingredients on the 
labels of the foods she is buying for her family. Yet on returning home she may flip on the 
television without any similar vigilance. Obsessed with material pollution, she is oblivious to 
information pollution. 



Nonmaterial pollution does exist. Most towns have regulations against nighttime noise, for 
example. At a loud party in the wee hours, it would be no surprise for the police to show 
up. Yet if there are no similar policies against light pollution, you might be able to light your 
driveway as brightly as technologically possible without concern that your neighbors would 
call the police. 



Desktop data glut has been described as information pollution, that makes foragers likely to 
'hunt elsewhere' (Nielsen, 2003). Shenk was the first to call it smog: 'Data smog gets in the 
way; it crowds out quiet moments, and obstructs much-needed contemplation. It spoils con- 
versation, literature, and even entertainment. It thwarts skepticism, rendering us less sophis- 
ticated as consumers and citizens. It stresses us out' (Shenk, 1997: 31). 



Yet to discuss information pollution is likely to contribute more of it. Also, it is subjective: 
information pollution fouls one person's thought, not everyone's waters. Above all, because 
one person's signal is another person's noise, it is much more difficult to agree on what is 
a common, noxious nuisance. The ethical questions raised differ from both those of free 
speech and those of clean air. Many of these concern whether information glut declares 
itself, and allows people to opt out (Greenfield 2004). The ethical questions can seem more 
culturally biased, as in what makes a society with many implicit forms of common sense 
much more pleasant than a nanny state with many explicit rules preventions. So how can 
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it be shown that glut fouls conviviality, which per Ivan lllich (1973) is all that which occurs 
socially without need for overt information? Such questions tie into ethics, politics, rhetoric, 
and other such fundamental categories in philosophy. It is a lot easier just to keep building, 
buying, and consuming more information feeds. 



3. Contextual Practices 



Third, then, this essay asks where current practices and common wisdom in the young field 
of interaction design show environmental awareness of any kind. Where might designed 
embodiment in ambient information actually increase awareness of the many issues raised? 
Such general questions seem warranted in a field where the pursuit of calming is a given 
(Weiser and Brown, 1997). It is the goal of this essay to provide intellectual background and 
future questions to that sensibility, and not to survey current practices in detail. How do 
context-sensitive design practices, which already emphasize symbolic environments, now 
make the step toward ambient environments? 



Interaction design has to be intrinsically environmental at some level because it expands the 
symbolic context of actions. Since its inception, the field has looked at data input and output 
in relationship to interface form and structure. It has looked at tasks in relationship to work- 
flows and organizations. It has looked at architectures of access. More recently it has looked 
at socially networked frames of reference, especially amid habitual and even institutional 
contexts. And now, as in the focus of this volume, it is looking at physical embodiment in 
urban space. 



Something about urban computing shifts the disciplinary emphasis from equipment to per- 
ceptions. It replaces a bias toward adding ever more tools and technological features with 
a quest toward a sensibility inflected toward maintaining some cultural commons. It differs 
from larger and necessary agendas in 'eco-tech,' that is, embedded information systems for 
improved resource economy, especially in building technology. It emphasizes proximity of 
particular people and annotation of particular places. It admits that knowledge of how to in- 
habit a city may be lost in the rush to personal entertainment. Indeed, distraction may be the 
state on which it must operate. The ethical issues at hand are those of distraction. 



As the literature of urbanism has long emphasized, individuality in places that are of inter- 
est to particular inhabitants produces a cumulative urban experience, such as the experience 
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of districts. The personalization of mobile media feeds does not produce this cumulative 
experience. This has often been understood though psychogeography. Physical experience 
introduces aspects of orientation, memory, and territory, largely because the body imposes 
a schema on space. While these do increase the sense of being in the world, the question 
arises whether they reinforce or interfere with existing senses of embodiment and environ- 
ment. On the other hand, new techniques of embedding, touch, and diversified display for- 
mats introduce a new domain of using the world itself as an access structure. In contrast to 
more innate relationships of body and memory, this new domain invites a more exploratory 
and less territorial approach to spatial mental mapping. That is because the world is now so 
much more overlaid and annotated with cultural productions of different spatial ordering. 
This much-championed hybrid introduces challenges of 'findability' (Morville, 2004) to which 
disciplinary approaches are quite different from the conventional literature on cognitive 
mapping. Thus new relations of embodiment, tangible interface, and ambient information 
increase the environmental implications of interaction design in two complementary ways. 



On the one hand, the discipline could increase its concern for shifts in focal attention. Early 
interaction design carefully studied focus amid tasks. Pervasive computing began from the 
introduction of peripheral attention (Weiser and Brown, 1997). The ambient is a continuum 
of possibilities for focal shift. Information theorists contend that complexity and emergence 
become central to any study of awareness amid ambient, noisy, glutted communications. 
'When information is understood as a process rather than a product, the line separating it 
from noise is difficult to determine,' Taylor explained. 'The interplay of noise, which is inform- 
ative, and information, which is noisy, creates the conditions for emerging complexity, which 
is the pulse of life' (Taylor, 2001: 123). 



On the other hand, the need to tune and personalize one's range of focal possibilities be- 
comes a central strategy of information anxiety management. This challenge was evident 
to Wurman twenty years ago, but it has accelerated, and in doing so it has implicated more 
contexts. The discipline might now read 'almost unconsciously' as 'embodied, haptically- 
oriented, socially cued.' One key to the ambient is the increasing belief that much mental 
activity does not rise to the level of conscious deliberation. 



Thus the overall social and practical contexts of action have become much more central to 
interaction design, and often surpass matters of mechanical or workflow-oriented usability 
as the key success factor. This makes the field more design-oriented. What one chooses to 
take on as subject matter matters as much as how efficiently that is engineered. For subject 
matter beyond the graphical user interface, the discipline works on product-service systems, 
neighborhood technologies, material and energy flow representation, and more. Because 
waste is almost intrinsically ugly, exposing and correcting it is a cultural, and not merely 
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moral or economic act. Environmental description, advocacy, and education allow more 
people to understand and shape such policy. Criticism suggests that these are cultural acts 
Hopefully this issue provides a review of many such projects. To see the change, one need 
only consult a CHI proceedings of ten years ago, when it was widely thought that only the 
virtual would count. 



4. Toward New Genres 



Fourth, the invitation here is to develop terms and criteria by which to give more discern- 
ment and design authority to these implicitly environmental computing practices. And the 
philosophical question is whether those design experiences cultivate (for a highly networked 
generation, many of whom seldom played outdoors as children) a sense of connection to the 
physical world. 



The first step is to demonstrate that locality is more than location. Proximate networks and 
not just GPS coordinates become the basis for locative media work. Locality and participa- 
tion seem key to many an emerging green economics. How does interaction design become 
valuable to this? 



Orientation to the physical everyday came first to history and letters. In the humanities, 
environmental criticism generally has origins in literary studies. Most university courses on 
environmental criticism concern literature first, and perhaps the high visual arts. Even with- 
out 'media,' a traditional culture created its environment mythologically; modern environ- 
mentalism might trace its origins to Virgil. Literature can blur distinctions among mythology, 
politics, environment, and perception. Was Mark Twain, educated as he was by the river, an 
early environmentalist? 

However, in recent decades, literary studies have often divided over environmental criticism. 
To the relativistic language games of postmodernity, any insistence on worldly referents 
seems naive, and any insistence on a master narrative (e.g., planetary change) demands op- 
position. Even spatial disciplines such as architecture long followed this literary lead. A bet- 
ter lead, however, appeared in cultural landscape studies. These ask how a society's values 
may be known as much from its settlement patterns as its literature or objects of art. Much 
of this has been directed to the macro scale, and sprawl, especially in America. But where it 
crosses with social history of technology, a useful genre of environmental criticism emerges. 
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For example, this is evident in the parallels commonly drawn between pervasive computing 
and early electrification. It is the ethnographic perspective of some of this work that makes 
it relevant to interaction design. For example McCarthy's 2001 study of ambient television 
looks at the influence of information pollution (our term, not hers) on gender roles and the 
spaces of conviviality and travel. As more kinds of ambient information technology become 
similarly normative, and no longer principally a technical challenge, surely they invite such 
similar critique. 



This is mainly a matter of building a culture. That perhaps begins with awareness of con- 
text. Only with a conversation that is accumulated and perpetuated does the larger body of 
work acquire enough coherence to warrant criticism. The word 'criticism' implies familiarity 
with the issues, and how to reconsider what you have been thinking anyway. For instance 
the original Macintosh was described as the first human-computer interface that was good 
enough to criticize. With design for embodied, tangible, and diversely displayed information, 
is pervasive computing at last good enough to criticize? 



One of the must fundamental sets of categories for critique has been introduced by Borg- 
man (1999) as information 'about,' 'for,' or 'as' the world. Information 'about' the world 
characterizes much pre-modernity. Even biologists concentrated on descriptions. Information 
'for' the world implies also some generative understanding, and is the main work of design. 
Modernists are notoriously spotty at seeing how things are because they are so busy im- 
agining how else things might become. That begins an environmental disconnection. Then 
there is information 'as' the world, 'a deluge that threatens to erode, suspend, and dissolve 
its predecessors' (Borgman, 1999: 2). Herein lies the main environmental disconnection. Me- 
dia-saturated people somehow become reality-challenged. Realism involves making sense of 
information types, especially in relationship to unmediated objects, structures, and surround- 
ings, especially those that cannot be turned off, put away, or ignored. 



Here let us limit the scope to information in environments, neither just describing them 
elsewhere like stories and maps, nor becoming them virtually like games and cinema, but, 
ubiquitous and embedded from within, engaged at annotating, operating, identifying, link- 
ing, recording, embellishing, polluting, populating, and filling the peripheries of what the best 
philosophers agree upon as reality. 
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Afterword 



Under philosophical shifts required by planetary change, environmental critics no longer 
oppose environment and culture, nor conflate environment with nature. Instead the critical 
challenge has becomes to recognize the origins of crisis, and prospects for change, in cultur- 
al attitudes toward any and all concepts of environment in which humanity plays a part. This 
includes built environment, managed biosphere, information sphere, and ambient cultural 
experience. How each of these disposes people to understand their place in the world now 
demands exposition. Bio-centric forms of environmentalism may always lead the way; yet to 
make such abstraction accessible to more imaginations, culturally-centric forms must also 
play a role. 



Information and environmental histories have not yet interwoven enough. Now, as informa- 
tion media become ubiquitous and interfaces become physically tangible and socially trans- 
parent, this separation may change. 'Environmental history of information' may no longer 
yield a null search on Google, as it did at this writing. 



Author's Biography 



Malcolm McCullough is associate professor of architecture at the University of Michigan. He 
has also served on the faculty of Carnegie Mellon and Harvard. He has written two widely- 
read books on architecture and interaction design - Digital Ground (2004) and Abstracting 
Craft (1996) - and has engaged in speaking and editorial review across a wide range of disci- 
plines, from urbanism through applied arts to interaction design. 



References 

Borgman, Albert. Holding on to Reality (Chicago: University of Chicago Press, 1999). 

Buell, Lawrence. From Apocalypse to Way of Life ; Environmental Crisis in the American Century 
(New York: Routledge, 2003). 



Greenfield, Adam. 'All watched over by machines of loving grace: Some ethical guidelines for user 



fibreculturejournal.org FCJ-136 175 



Toward Environmental Criticism 



experience in ubiquitous-computing settings.' Boxes and Arrows, www.boxesandarrows.com. 2004. 
Illich, Ivan. Tools for Conviviality. (New York: Harper and Rowe, 1973). 

McCarthy, Anna. Ambient Television: Visual Culture and Public Space (Durham: Duke University 
Press, 2001). 

Miller, Henry. The Air Conditioned Nightmare (San Francisco: New Directions, 1970). 

Morville, Peter. Ambient Findability (Sebastopol, CA: O'Reilly 2005). 

Mumford, Lewis. Technics and Civilization (New York: Harcourt, Brace, 1936). 

Nielsen, Jacob. 'Information Pollution.' Alertbox. www.useit.com. 11 August 2003, <http://www. 
useit.com/alertbox/20030811.html>. 

Owen, David. 'Green Manhattan— Why New York is the greenest city in the U.S.' The New Yorker. 
18 October 2004. 

Shenk, David. Data Smog - Surviving the Information Glut (San Francisco: Harper Collins, 1997). 

Taylor, Mark. C. The Moment of Complexity - Emerging Network Culture (Chicago: University of 
Chicago Press, 2001). 

Weiser, Mark, and John Seely Brown. 'The Coming Age of Calm Technology.' 1996. 15.08.2011. 
<http://nano.xerox.com/hypertext/weiser/acmfuture2endnote.htm>. 

Worster, D. 'Appendix: Doing Environmental History' The Ends of the Earth; Perspectives on Mod- 
ern Environmental History (New York: Cambridge University Press, 1988). 

Wurman, Richard Saul. Information Anxiety (New York: Doubleday, 1989). 




The LOCKSS System has the permission to collect, 
preserve and serve this open access Archival Unit 




The Fibreculture Journal is published under a Creative 
Commons, By Attribution-Non Commercial-No Derivative 



C) 



OPEN HUMANITIES PRESS 

The Fibreculture Journal is an Open Humanities Press Journal. 



176 



FCJ-136 



fibreculturejournal.org 



The Fibreculture Journal 



DIGITAL MEDIA + NETWORKS + TRANSDISCIPLINARY CRITIQUE 




FC3-137 AFFective Experience in Interactive Environments 



Jonas Fritsch 

Department of Aesthetics and Communication, Aarhus University 



Introduction 



Digital technologies in new interactive environments are radically affecting the way we experi- 
ence and make sense of the world. The advent of ubiquitous computing in particular has led to 
the development of advanced sensor technologies and microchips, moving the realm of com- 
puting from the desktop computer into broader contexts of interaction or interactive environ- 
ments (Weiser, 1991). It seems, however, that there has been a shift from the original vision of 
ubiquitous computing in the light of the possibilities that a ubiquitous computational infrastruc- 
ture has actually been shown to offer. Mark Weiser's initial ideal of disappearing computers, so 
the technology resides in the background to let us focus on more important things, is no longer 
the only way of working with ubiquitous computing and technologies. Among others, Yvonne 
Rogers has proposed a new agenda for the design of UbiComp technologies focused on the 
creation of engaging user experiences, where the technology does not disappear but is actively 
used to create particular experiences (Rogers, 2006). In line with this, Petersen insists that 
objects and environments embedded with interactive technologies are becoming remarkable 
objects of lifestyle and identity (Petersen, 2004). 



This tendency is currently being explored in interaction design, described by Winograd (1997) as 
a heterogeneous academic discipline concerned with designing digital artefacts, based on an 
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understanding of the relationship between humans and digital technology (Winograd, 1997: 
161). This understanding is subject to changes due to developments in digital possibilities, 
human needs, and situated social as well as physical contexts (Thackara, 2001). Consequent- 
ly, interaction design is forced to continually develop new theoretical, analytical, and practi- 
cal tools that may contribute to the design of new interactive environments. These allow us 
to experiment with how we might live with the technologies in the future. 



Designing for living with ubiquitous technologies implies consideration of the experiential 
qualities that come into play to form our experience of designed interactive environments 
(McCarthy and Wright, 2004). The way the experiences offered by ubiquitous, interactive 
environments are conceptualised in interaction design largely determines the outcomes of 
the design processes. It therefore becomes necessary for interaction design to develop an 
understanding of, and a vocabulary to describe, how interactive environments and technolo- 
gies might shape our situated affective experiences. 



This article introduces the notion of affect as it is developed in the philosophy of Brian Mas- 
sumi as a contribution to meeting this challenge. Massumi develops a philosophy of experi- 
ence which is particularly useful for investigating how ubiquitous interactive environments 
might engage us affectively. I will suggest here that this is primarily through the creation of 
fields of experience. The concept of affect itself is inherently relational. It describes the cor- 
relation between a person's ability to think and feel and the body's ability to act in the world. 
Taking Massumi's notion of affect as a starting point for the design of experience-oriented 
interactive environments forces us to focus on the affective tonalities and the relational 
events that might emerge through the interaction. 



Before entering the conceptual development, however, I take a step back to relate ubiquitous 
computing to recent developments in experience-oriented interaction design. This sets the 
scene for introducing Massumi's work on the concept of affect. Through an analysis of the 
interactive installation City Voices I develop an analytic understanding of how the installation 
changes our affective experience, bodily activation and capacitation in an interactive envi- 
ronment in a public setting. Following this analysis, I outline how taking the complexity of 
the concept of affect seriously in the design process can make it possible to sketch out new 
horizons for designing affectively engaging interactive environments. 
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Ubiquitous Computing and Experience-oriented Interaction 
Design 



There can be no doubt that Mark Weiser's The Computer for the 2P* Century has been semi- 
nal for setting the agenda for most of the work done when designing and conceptualising 
ubiquitous computing in industrial as well as research settings. His vision of calm technolo- 
gies that stay out of the way and disappear into the background, appearing only when and 
where we need them, has spurred a great many both practical and theoretical developments 
that make technologically enhanced living convenient and comfortable (Weiser, 1991; see 
Rogers, 2006 for a comprehensive project overview). Yet, in an article from 2006, Rogers 
argues that there is an enormous gap between the 'dream of comfortable, informed and ef- 
fortless living and the accomplishments of UbiComp research' (Rogers, 2006: 405). Instead, 
she proposes that the field of ubiquitous computing should broaden its scope and address 
other goals than the seamless integration of digital technologies in our everyday living. This 
leads to an agenda for designing UbiComp technologies for what she terms 'engaging user 
experiences' (Rogers, 2006: 406): 



The 'excitement of interaction' that Weiser suggested forsaking in the pursuit of a vision of 
calm living should be embraced again, enabling users, designers and researchers to partici- 
pate in the creation of a new generation of user experiences that go beyond what is current- 
ly possible with our existing bricolage of tools and media. We should be provoking people in 
their scientific, learning, analytic, creative, playing and personal activities and pursuit. (Rog- 
ers, 2006: 418) 



The ideas presented by Rogers resonate with a general concern within the respective fields 
of digital art, digital aesthetics and interaction design. Bolter and Gromala (2004) use the 
concepts of transparency and reflectivity to describe two strategies in interface design. The 
aesthetics of the former is concerned with the disappearance of the media whilst the aes- 
thetics of the latter invites and forces an active reflection on the part of the user in relation 
to the media. Recently, Munster (2006) has argued for an active understanding of the sen- 
sational and experiential aspects of interacting with new media. These should not be hidden 
away in the analysis and design, but further explored in facilitating new relations between 
people and digital technology: 



Far from disposing of the senses, new media, whether commercially realized or 
experimentally proposed, point toward the synaesthetic disruptions and reconfigura- 
tions of bodily capacities and functions that might be made possible by digital tech- 
nologies. (Munster, 2006: 19) 
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Interaction design today is investigating theoretical and practical ways to uncover new 
contexts of use, along with new conceptualisations of users to design for. People are not 
always rational or striving for efficiency or transparency in their interaction with technol- 
ogy; happiness, reflection, provocation, and desire also play a functional role in the design 
of new, remarkable interfaces, products and interactive environments (Dunne, 1999). The 
shift is also manifested in the rise of the experience-oriented domain of interaction design. 
Among others, McCarthy and Wright (2004) argue that experience-oriented interaction 
design can lead to the creation of innovative and creative forms of digital technology with 
human experience as the turning point. 



A common concern in interaction design today can thus be said to be the articulation of a 
new vocabulary for addressing experiential concerns in design work along with an explora- 
tion of what that means for the future design of ubiquitous technologies. In this article I will 
argue that the concept of affect as developed by Brian Massumi can be beneficial for ad- 
dressing and problematising this concern. The concept of affect is not foreign to recent de- 
velopments in Human-Computer Interaction (HCI) and interaction design (see Fritsch, 2009 
for a comprehensive overview). However, the Massumian notion of affect has yet to find its 
way into the discourse. In the following, I will introduce the vocabulary proposed by Mas- 
sumi and argue that it might contribute to the challenges of designing affectively engaging, 
experience-oriented, interactive environments. 



AFFective Experience 



Brian Massumi works at the intersection of philosophy, cultural theory, art, interactive archi- 
tecture, politics and design. In his writings, he proposes a philosophy of experience which is 
inspired by radical empiricism primarily as this is exemplified in the work of William James. 
In radical empiricism, focus is on the relations between particulars. Relations that have the 
same ontological and epistemological status as the particulars themselves. Massumi talks 
about his own philosophical project as a form of expanded radical empiricism in which 'ex- 
perience is an additive "form of transition", a continued motion of intersecting process lines: 
a co-motion (commotion) of mutual non-exclusion' (Massumi, 2002: 213). 



According to Massumi, experience never stops. It can never be confined to a static, objective 
description. Experience is dynamic. However, it is possible to look into the formation of expe- 
rience - how experience works, what makes us experience - bringing into play the relational 
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complexity advocated by a radical empiricist philosophical approach. To arrive at a descrip- 
tion of these general conditions of emergence of experience, Massumi argues that it is nec- 
essary to investigate that which comes before an actual experience, the preindividual. The 
notion of the preindividual is taken from French philosopher Gilbert Simondon to denote that 
which has not yet taken any determinate form, but modulates the formation of experience 
and the potential for action in a given situation. The preindividual is felt and experienced 
without being registered consciously. This does not mean, however, that it does not have an 
important impact on what we experience consciously. 



Thinking with Massumi in relation to interaction design offers a way to describe the dynam- 
ics that govern the transition from the non-conscious or preindividual dimensions of the ex- 
periential to the experienced. Thinking this way involves thinking about the workings of what 
Massumi in several places refers to as the field of experience (Massumi, 2002: 4). The field of 
experience addresses the potential for experience as much as the outcome, which is a quali- 
fied experience. Massumi specifically addresses this field through the notion of affect and 
the role it plays in our experiences in-the-making [1]. However, working with affect demands 
conceptual caution. It does not offer definite answers but rather what Massumi himself re- 
fers to as a 'field of questioning': 



The notion of affect does take many forms, and you are right to begin by emphasiz- 
ing that. To get anywhere with the concept, you have to retain the manyness of its 
forms. It's not something that can be reduced to one thing. Mainly because it's not 
a thing. It's an event, or a dimension of every event. What interests me in the con- 
cept is that if you approach it respecting its variety, you are presented with a field of 
questioning, a problematic field where the customary divisions that questions about 
subjectivity, becoming or the political are usually couched in do not apply. (Massumi 
and McKim 2009; 1) 

When developing his notion of affect, Massumi takes as a starting point Gilles Deleuze's 
reception of Spinoza's idea of affect. This is affect as something which can be defined in 
terms of the capacity to affect or be affected. This definition makes it possible to think of af- 
fect in active terms, avoiding a strict focus on passive reception. Massumi stresses that the 
importance of the capacity to affect and the capacity to be affected are two facets of the 
same event. Capacitation here refers to a preindividual and virtual potential for action expe- 
rienced affectively which may or may not translate into an actual(ised) action line. The power 
to affect and be affected governs a transition where a body passes from one state of ca- 
pacitation to an augmented or diminished stat of capacitation, a transition which is felt. The 
felt quality of a given experience is that which characterises the feeling of the transition as 
the body moves from one power of existence to another, which is separate from the actual 
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capacitation understood as potential for action. Massumi argues that positive affects are 
those that make us feel alive and act in the world. Negative affects have the opposite effect, 
reducing our possible activity in the world and making this reduction felt. 



In the Massumian affective account, a body is defined by what capacities it carries from step 
to step. The charge of affect is not something fixed and it plays out differently in any given 
situation. Instead of working with pre-determined and static notions of the experiential, and 
in keeping with the radical empiricist approach, focusing on affective experience stresses 
the relations that occur in the middle of a field of experience. It is necessary to understand 
relational events that play out differently every time, take up the past differently, creating 
new potentials for the future. Massumi underlines that an account of affect has to '...directly 
address forms of experience, forms of life, on a qualitative register' (Massumi and McKim, 
2009: 1). The affective position accounts for the way an experiential field - a potentialising 
set of conditions of emergence - effects our preindividual but felt relation to the world. In the 
following section this will be explored through the concepts of affective tonality and proprio- 
ception. This will allow a more elaborate description of the workings of the preindividual. 



AFFective Tonality and Proprioception 



Massumi develops the concept of affective tonality from Alfred N. Whitehead who argues 
that we always apprehend the affective tone of a given situation. The affective tonality is 
not something residing in either the subject or the object. It emerges in the actual encounter 
while simultaneously modulating the occurrence (2): 



What Whitehead calls affective tonality is something we find ourselves in, rather than 
finding in ourselves. An embracing atmosphere that is also at the very heart of what 
happens because it qualifies the overall feel. (Massumi, 2007; 82) 

According to Massumi, the capacity to be affected and to affect is formed by the affective 
tonality of a given situation. However, rather than determining our lines of action the affec- 
tive tonality is tied to tendencies that will always play out differently and event-fully in every 
situation. Some tendencies will actualise and some will remain in the virtual, but they still 
remain part of the really felt affective experience. Massumi emphasises that the body's ten- 
dencies are also like this. They are activated as the body moves into and through situations. 
This leads to a relational complexity that provides conditions of emergence for more quali- 
fied experiences: 
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...the body is that region of in-mixing from which subjectivity emerges. It is the com- 
ing together of the world, for experience, in a here-and-now prior to any possibility of 
assigning categories like subject or object. (Massumi, 2009: 3) 

Starting from affect in the description of the body is 'an invitation for an indefinitely con- 
structive thinking of embodied, relational becoming' (Massumi and McKim , 2009: 2). The 
tendencies in the body take many forms 'as instincts, inclinations, teeming feelings, masses 
of memories, conscious and nonconscious' (Massumi and McKim, 2009: 2). The question 
becomes one of exploring how this crowding is moved into a constitution, as a qualified ex- 
perience. This question is ultimately a question of the emergence and formation of a subject: 
'The subject of an experience emerges from a field of conditions which are not that subject 
yet, where it is just coming into self (Massumi and McKim, 2009: 3). 



The notion of proprioception is also crucial to the Massumian experiential vocabulary. It is 
defined as the sensibility proper to the muscles and ligaments, as opposed to tactile sensi- 
bility (which is 'exteroceptive') and visceral sensibility (which is 'interoceptive'). Propriocep- 
tion is preindividual yet actualised, really felt bodily but not consciously reflected upon: 



Proprioception effects a double translation of the subject and the object into the 
body, at a medium depth where the body is only body, having nothing of the putative 
profundity of the self nor of the superficiality of external encounter. (Massumi 2002: 
59) 

Massumi argues that proprioception is always a primary phase in every sensation. It is 
the body's self-feeling of a posturing for movement, nonconscious by nature, barely active 
before unfolding into action. It can only be felt, lived out, transduced into an ensuing action- 
line. Proprioception is an enactive awareness of the body's own movement, affecting the 
body's actual unfolding. The body's capacity for action or activation is therefore this bodily 
field of potential, where affect is what connects the virtual series with their actual event-ful 
unfolding as real, preindividual tendencies forming an experiential field which might trans- 
form into personally qualified experiences (Massumi, 2002). In sum, proprioception actively 
works in and forms an experiential background. 



A Massumian approach to working with experience-oriented interaction design would ac- 
count for the fields of experience offered by interactive, ubiquitous environments. It would 
uncover how they work affectively, offering particular conditions of emergence for the inter- 
action. It would take into consideration the affective tonality of these environments, in terms 
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of how they might activate and capacitate bodies as they move into and through interac- 
tive situations. It would describe the relational complexity of the events forming and being 
formed by the affective experience of moving through and with an interactive environment 
In all this, an affective point of view necessitates looking into potential situations of emer- 
gence and the re-conditionings of the emerged in relation to becoming: 



Conditions of emergence are one with becoming. Re-conditionings of the emerged 
define normative or regulatory operations that set the parameters of history (the pos- 
sible interactions of determinate individuals and groups). (Massumi 2002: 10) 



In the following, through the analysis of the interactive installation City Voices, I will try to 
show how the preceding vocabulary can be used to describe the affective experience offered 
by a particular interactive environment. 



City Voices as a Field oF Experience 



City Voices was a citizen involvement project in the form of an exhibition. It focused on gath- 
ering values and ideas for the design of the future Multimedia House, a 28,000 m2 building 
with surrounding areas, to be built at the harbour in Aarhus, Denmark from 2012-2015. The 
exhibition was designed by the architect and interaction design firm KOLLISION (www.kolli- 
sion.dk). 




Figure 1: City Voices at the main Library (left) and at the Aarhus 
Centre for Contemporary Art (right) 
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The project consisted primarily of two interactive installations; one exhibited in the main 
library of Aarhus and one in the Aarhus Centre for Contemporary Art (Fig. 1). Each installa- 
tion involved an interactive table on wheels. When one moved the tables, a map displayed 
on a table top screen was panned, making the table a peephole into a larger digital map. A 
compass was wired to the tables, so the digital map would also relate to the physical place- 
ment of the table in terms of N-S-E-W directionality. The user would navigate the digital map 
by moving the table around in the physical room. It was possible to choose between three 
different digital maps to be displayed on the table; a local map of Aarhus, a map of Den- 
mark, and a world map. On the table, parts of the maps would be displayed that could be 
navigated by moving the table. On the wall the user could see the full map and the selection 
of the map presented on the table (Fig. 1., Fig. 2). 



On each of these maps a number of scenarios were visualised on the interface. Each one 
contained a voice from the future discussing a personal experience in relation to the finished 
Multimedia House. The idea was to inspire the users to voice their own opinions and dis- 
cuss their own experiences and expectations. A microphone on the table allowed the users 
to record their own scenario and leave it for the next user to encounter. During the process 
the users' scenarios would be accumulated for all to hear along with the original scenarios. 
These user scenarios could be accessed through the two interactive tables and the project 
website. 




Figure 2: People interacting with the table (left) and a close-up of the map 
displayed on the table, which you could only navigate by moving around in 
the physical room (right). 
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City Voices uses ubiquitous technologies in the design of on interactive environment experi- 
menting with the affective tonality of the public setting. The installation is a post-desktop 
interface encouraging bodily movement in order to explore the digital content and physical 
constraints of the interaction with the system. The interactive tables are not facilitating the 
search for information or disappearing into the background. Instead, they actively foster 
curiosity and engaged exploration of the content of the exhibition by providing new means of 
interaction with the underlying digital system, contributing to a different affective experience 
of the technological setup. When you interact with City Voices, you are immediately engaged 
in the exploration of an experiential field activated by the installation. The need to move the 
table around in the exhibition space activates you proprioceptively. The table itself becomes 
an interface between your body and the system; your bodily movement is necessary to navi- 
gate the system, and this alters the affective tonality of the interaction. You explore actual 
movements in space, choreographed or performed by the installation. Simultaneously, the 
movement is registered proprioceptively and by the system itself processing the data. There 
is a doubling of a registering of movement. This is both visualised in the system and felt in 
the interaction, or more accurately occurring in-between as part of the affective tonality. 



The interactive tables and the coupling of the digital content and physical interaction with 
their co-emerging spatial realities profoundly alter the affective tonality. You interact with the 
table in a room based on a partial map of the city visualised on the screen on the table. Si- 
multaneously you also orient yourself towards the larger map on the wall. Trying to navigate 
all three scales of interaction smoothly is almost impossible. You need to continuously stop 
and situate yourself in relation to the scale with which you are trying to connect. Attention 
shifts between interacting with the table, orienting yourself towards the visualisations on 
the walls and moving about in the room creating a complex relational space of interaction. 
Interestingly, though, the installation is rarely used by only one user. More people can easily 
gather in the room or around the screen, creating the scene for social interaction which in 
turn affects coordination of the movements and explorations. 



City Voices seeks to engage people in the future planning of the city landscape by provid- 
ing cues as forms of scenarios. These are meant to trigger a participation in the project that 
involves leaving a voice message. It remains possible that the interaction with the setup 
might in the future effect a change in the cityscape when the Multimedia House is built. This 
is capacitation on a different experiential level, yet it still feeds into the affective experience 
within a defined interactive setup. At the same time, the engagement with City Voices goes 
beyond the immediate and already doubled spatio-temporality of the interaction with the 
installation here-and-now. It facilitates a possible long-term relationship or more correctly a 
possible re-occurrence or retro-activation of the interactions preserved in the data collected 
and presented both on the two interactive tables and on the shared website. 
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What exists virtually as part of the future of the cityscape is activated and partly actualised 
through City Voices as relational events. The installation tries to make existing and new rela- 
tions felt differently. City Voices, like the name suggests, gives people a 'voice', it creates the 
potential for strong affective ties by capacitating people, giving them an opportunity to voice 
their opinions and feelings. In this way, the installation 'bridges' the affective with emotivity, 
emotions, narratives and communication. All these thresholds of the experiential continuum 
are somehow activated and distributed in the physical, social and affective spaces the instal- 
lation creates. 



In this analysis, the concept of affect and the resulting vocabulary work to describe the 
experiential field that is offered by the particular interactive setup City Voices. People are 
invited to activate and affect the setup; in turn, there is an affectation through movements, 
interactions and reflections. How this affectation or activation is lived or felt depends on the 
specific situation; the relational events play out differently every time. However, the en- 
gaged exploration and the experiential field facilitated by the interaction design does offer 
the potential (and an invitation) for activation and capacitation. In this respect. City Voices 
is an experimental exploration of how a designed urban interactive environment might take 
a wholly relational and interactional form by altering the affective tonality of interaction, in 
a public setting, and with digital and ubiquitous technologies. The technological exploration 
investigates new forms of proprioceptive engagement in the interactional events. Finally, the 
installation provides conditions of emergence and cues for re-conditions of the emerged that 
might or might not translate into more stable or qualified forms of becoming. 



Concluding Remarks 



The analysis of City Voices can be seen as an example of how the design of an interactive 
environment - and the analysis of the experiential field it offers - might be conceptualised 
in affective terms. Rather than disappearing, the digital technology in City Voices is used 
actively to engage users in the exploration of the installation, thus experimenting with the 
affective tonality of the cityscape, the interactive situation and the possible relational events 
that might emerge. Ubiquity in this setting results in a complex interaction design of a pos- 
sible space stirring with activation and affective capacitation. 



The Massumian understanding of affect as developed in this article has the potential to offer 
important insights concerning the theoretical foundations for practical design experiments 
with experience-oriented interaction design. However, in keeping with the Massumian philos- 
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ophy of experience, it is important to stress that affective experience cannot and should not 
be used as a blueprint for design. Arguably, the notion of affect with its attached vocabulary 
raises more questions than it answers. This is exactly the point; the concept of affect offers 
to interaction design, as it does to philosophy and cultural theory, a 'field of questioning,' a 
problematic field 'starting in the middle with the full complexity and dynamics of fields of 
experience.' This, to me, seems to be the right place to start any experience-oriented design 
process. 
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Notes 



[1] Importantly, in a Massumian vocabulary affect is distinct from emotions or feelings that 
can be seen as a recognized outcomes of affect (Massumi, 2002: 61). 



[2] In line with this Erin Manning argues that the affective tone can also be described as the 
relational concernedness of the emerging world, again highlighting the radical empiricist ap- 
proach and the relational as integral to a description of the affective experience (Manning, 
2009). 
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